Bug 44881

Summary: pci_find_upstream_pcie_bridge() warning
Product: Drivers Reporter: John Wehin (john.wehin)
Component: PCIAssignee: Alex Williamson (alex.williamson)
Status: RESOLVED CODE_FIX    
Severity: normal CC: admin, akamaka1024+bugtrackerkernelorg, alan, alex.williamson, Andreas.Klauer, andreas.sturmlechner, andrey, anickname, animainvendita, bjorn, dgo, ganastasiouGR, ingo.theiss, jlec, john.wehin, kalin, lsching17, milos.kaurin, nodenet, reuben-kernel-bugzilla, robb
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 3.11 Subsystem:
Regression: Yes Bisected commit-id:
Attachments: debug patch
Disable VT-d in presence of these devices
Handle unadvertised PCIe bridges
With this patch applied and no PCI logged
With this patch applied and PCI card logged

Description John Wehin 2012-07-17 10:26:47 UTC
[1.] WARNING: at drivers/pci/search.c:44 pci_find_upstream_pcie_bridge+0x56/0x6f()

[2.] https://bugzilla.redhat.com/show_bug.cgi?id=755943
kernel version 3.1.1, 3.3.8, 3.4.4 and 3.5.0-rc7

[ 0.314976] Unpacking initramfs...
[ 0.542862] Freeing initrd memory: 17964k freed
[ 0.544588] DMAR: No ATSR found
[ 0.544616] IOMMU 0 0xfed90000: using Queued invalidation
[ 0.544617] IOMMU 1 0xfed91000: using Queued invalidation
[ 0.544618] IOMMU: Setting RMRR:
[ 0.544627] IOMMU: Setting identity map for device 0000:00:02.0 [0xaf800000 - 0xcf9fffff]
[ 0.546932] IOMMU: Setting identity map for device 0000:00:1d.0 [0xae0a9000 - 0xae0d3fff]
[ 0.546950] IOMMU: Setting identity map for device 0000:00:1a.0 [0xae0a9000 - 0xae0d3fff]
[ 0.546959] IOMMU: Prepare 0-16MiB unity mapping for LPC
[ 0.546964] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.547047] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[ 0.547060] ------------[ cut here ]------------
[ 0.547065] WARNING: at drivers/pci/search.c:44 pci_find_upstream_pcie_bridge+0x66/0x80()
[ 0.547066] Hardware name: To Be Filled By O.E.M.
[ 0.547067] Modules linked in:
[ 0.547069] Pid: 1, comm: swapper/0 Not tainted 3.5.0-0.rc7.git0.1.fc16.x86_64 #1
[ 0.547070] Call Trace:
[ 0.547075] [<ffffffff8105927f>] warn_slowpath_common+0x7f/0xc0
[ 0.547077] [<ffffffff810592da>] warn_slowpath_null+0x1a/0x20
[ 0.547079] [<ffffffff81300986>] pci_find_upstream_pcie_bridge+0x66/0x80
[ 0.547082] [<ffffffff814e4e9c>] intel_iommu_device_group+0x7c/0x100
[ 0.547084] [<ffffffff814da85a>] add_iommu_group+0x3a/0x60
[ 0.547086] [<ffffffff814da820>] ? bus_set_iommu+0x50/0x50
[ 0.547090] [<ffffffff813b7be6>] bus_for_each_dev+0x56/0x90
[ 0.547091] [<ffffffff814da810>] bus_set_iommu+0x40/0x50
[ 0.547094] [<ffffffff81d3549e>] intel_iommu_init+0xa9b/0xb2a
[ 0.547097] [<ffffffff81042690>] ? free_init_pages+0x130/0x1c0
[ 0.547099] [<ffffffff81cf7b3c>] ? md_run_setup+0x8b/0x8b
[ 0.547102] [<ffffffff81cfe002>] ? memblock_find_dma_reserve+0x13d/0x13d
[ 0.547104] [<ffffffff81cfe015>] pci_iommu_init+0x13/0x3e
[ 0.547106] [<ffffffff8100203f>] do_one_initcall+0x3f/0x170
[ 0.547108] [<ffffffff81cf6d7c>] kernel_init+0x14e/0x1d2
[ 0.547110] [<ffffffff81cf6610>] ? do_early_param+0x86/0x86
[ 0.547113] [<ffffffff8161a8a4>] kernel_thread_helper+0x4/0x10
[ 0.547114] [<ffffffff81cf6c2e>] ? start_kernel+0x3c4/0x3c4
[ 0.547116] [<ffffffff8161a8a0>] ? gs_change+0x13/0x13
[ 0.547121] ---[ end trace a307b3ed40206b48 ]---
[ 0.550925] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni)
[ 0.551079] Initialise module verification
[ 0.551105] audit: initializing netlink socket (disabled)
[ 0.551113] type=2000 audit(1342502443.429:1): initialized
[ 0.564296] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.565436] VFS: Disk quotas dquot_6.5.2
[ 0.565465] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.565683] msgmni has been set to 15002
[ 0.565716] Key type crypto registered

[3.] kernel, driver, IOMMU, PCI, ISA, LPC, PCIE


[4.1.] Linux version 3.5.0-0.rc7.git0.1.fc16.x86_64 (John@devel.f2f.local) (gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) ) #1 SMP Tue Jul 17 00:52:51 MSK 2012

[8.1.]
Linux devel.f2f.local 3.5.0-0.rc7.git0.1.fc16.x86_64 #1 SMP Tue Jul 17 00:52:51 MSK 2012 x86_64 x86_64 x86_64 GNU/Linux
 
Gnu C                  4.6.3
Gnu make               3.82
binutils               2.21.53.0.1
util-linux             2.20.1
mount                  support
module-init-tools      3.16
e2fsprogs              1.41.14
xfsprogs               3.1.5
pcmciautils            018
quota-tools            4.00-pre1.
PPP                    2.4.5
Linux C Library        2.14.90
Dynamic linker (ldd)   2.14.90
Procps                 3.2.8
Net-tools              1.60
Kbd                    1.15.2
Sh-utils               8.12
wireless-tools         29
Modules Loaded         ppdev parport_pc lp parport fuse ebtable_nat ebtables lockd xt_CHECKSUM bridge stp llc be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi tpm_bios ip6t_REJECT ip6t_ipv6header nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_owner xt_state nf_conntrack snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd vhost_net macvtap macvlan tun i2c_i801 coretemp soundcore snd_page_alloc serio_raw uinput r8169 mii kvm_intel kvm microcode binfmt_misc sunrpc crc32c_intel ghash_clmulni_intel i915 drm_kms_helper drm i2c_algo_bit i2c_core video

[8.2.]
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31235 @ 3.20GHz
stepping        : 7
microcode       : 0x28
cpu MHz         : 1600.000
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 6385.41
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

[8.3.]
ppdev 17635 0 - Live 0xffffffffa0237000
parport_pc 28082 0 - Live 0xffffffffa0311000
lp 17874 0 - Live 0xffffffffa0294000
parport 40419 3 ppdev,parport_pc,lp, Live 0xffffffffa0222000
fuse 77964 3 - Live 0xffffffffa04e0000
ebtable_nat 12807 0 - Live 0xffffffffa04cf000
ebtables 30688 1 ebtable_nat, Live 0xffffffffa04d7000
lockd 89127 0 - Live 0xffffffffa04b8000
xt_CHECKSUM 12549 0 - Live 0xffffffffa04a5000
bridge 91137 0 - Live 0xffffffffa048d000
stp 12823 1 bridge, Live 0xffffffffa0488000
llc 14090 2 bridge,stp, Live 0xffffffffa045f000
be2iscsi 81635 0 - Live 0xffffffffa0473000
iscsi_boot_sysfs 15641 1 be2iscsi, Live 0xffffffffa0424000
bnx2i 54521 0 - Live 0xffffffffa0464000
cnic 58779 1 bnx2i, Live 0xffffffffa044f000
uio 19028 1 cnic, Live 0xffffffffa03ad000
cxgb4i 32909 0 - Live 0xffffffffa0445000
cxgb4 108336 1 cxgb4i, Live 0xffffffffa0429000
cxgb3i 32972 0 - Live 0xffffffffa040b000
libcxgbi 56508 2 cxgb4i,cxgb3i, Live 0xffffffffa0415000
cxgb3 155459 1 cxgb3i, Live 0xffffffffa03e4000
mdio 13398 1 cxgb3, Live 0xffffffffa03d4000
ib_iser 38039 0 - Live 0xffffffffa03d9000
rdma_cm 41898 1 ib_iser, Live 0xffffffffa03c8000
ib_cm 41692 1 rdma_cm, Live 0xffffffffa03bc000
iw_cm 18176 1 rdma_cm, Live 0xffffffffa03b6000
ib_sa 28417 2 rdma_cm,ib_cm, Live 0xffffffffa0397000
ib_mad 46392 2 ib_cm,ib_sa, Live 0xffffffffa03a0000
ib_core 73874 6 ib_iser,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mad, Live 0xffffffffa0383000
ib_addr 13745 1 rdma_cm, Live 0xffffffffa037e000
iscsi_tcp 18333 0 - Live 0xffffffffa0374000
libiscsi_tcp 24128 4 cxgb4i,cxgb3i,libcxgbi,iscsi_tcp, Live 0xffffffffa035e000
libiscsi 50527 8 be2iscsi,bnx2i,cxgb4i,cxgb3i,libcxgbi,ib_iser,iscsi_tcp,libiscsi_tcp, Live 0xffffffffa0366000
scsi_transport_iscsi 57474 8 be2iscsi,bnx2i,libcxgbi,ib_iser,iscsi_tcp,libiscsi, Live 0xffffffffa034e000
tpm_bios 13528 0 - Live 0xffffffffa0349000
ip6t_REJECT 12939 2 - Live 0xffffffffa0344000
ip6t_ipv6header 12590 2 - Live 0xffffffffa033f000
nf_conntrack_ipv6 14334 24 - Live 0xffffffffa0331000
nf_defrag_ipv6 18221 1 nf_conntrack_ipv6, Live 0xffffffffa0327000
ip6table_filter 12815 1 - Live 0xffffffffa030c000
ip6_tables 26845 1 ip6table_filter, Live 0xffffffffa0304000
xt_owner 12498 0 - Live 0xffffffffa02ff000
xt_state 12578 24 - Live 0xffffffffa0232000
nf_conntrack 83564 2 nf_conntrack_ipv6,xt_state, Live 0xffffffffa02a0000
snd_hda_codec_hdmi 36157 1 - Live 0xffffffffa0218000
snd_hda_codec_realtek 77529 1 - Live 0xffffffffa02eb000
snd_hda_intel 37532 2 - Live 0xffffffffa02e0000
snd_hda_codec 126412 3 snd_hda_codec_hdmi,snd_hda_codec_realtek,snd_hda_intel, Live 0xffffffffa02c0000
snd_hwdep 13515 1 snd_hda_codec, Live 0xffffffffa029b000
snd_seq 64808 0 - Live 0xffffffffa0246000
snd_seq_device 14129 1 snd_seq, Live 0xffffffffa022d000
snd_pcm 97362 3 snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec, Live 0xffffffffa027b000
snd_timer 28815 2 snd_seq,snd_pcm, Live 0xffffffffa023d000
snd 79280 13 snd_hda_codec_hdmi,snd_hda_codec_realtek,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_seq,snd_seq_device,snd_pcm,snd_timer, Live 0xffffffffa0203000
vhost_net 33675 0 - Live 0xffffffffa01e1000
macvtap 18183 1 vhost_net, Live 0xffffffffa01f9000
macvlan 18644 1 macvtap, Live 0xffffffffa00f9000
tun 22804 2 vhost_net, Live 0xffffffffa01da000
i2c_i801 17765 0 - Live 0xffffffffa01f3000
coretemp 13524 0 - Live 0xffffffffa0165000
soundcore 14484 1 snd, Live 0xffffffffa01eb000
snd_page_alloc 18101 2 snd_hda_intel,snd_pcm, Live 0xffffffffa016f000
serio_raw 13375 0 - Live 0xffffffffa0160000
uinput 17606 0 - Live 0xffffffffa015a000
r8169 61461 0 - Live 0xffffffffa0149000
mii 13527 1 r8169, Live 0xffffffffa016a000
kvm_intel 132323 0 - Live 0xffffffffa0259000
kvm 408129 1 kvm_intel, Live 0xffffffffa0175000
microcode 23435 0 - Live 0xffffffffa0067000
binfmt_misc 17431 1 - Live 0xffffffffa005e000
sunrpc 254796 2 lockd, Live 0xffffffffa0109000
crc32c_intel 12858 0 - Live 0xffffffffa0101000
ghash_clmulni_intel 13180 0 - Live 0xffffffffa0015000
i915 510236 2 - Live 0xffffffffa007b000
drm_kms_helper 40075 1 i915, Live 0xffffffffa0070000
drm 259231 3 i915,drm_kms_helper, Live 0xffffffffa001d000
i2c_algo_bit 13250 1 i915, Live 0xffffffffa0006000
i2c_core 34044 5 i2c_i801,i915,drm_kms_helper,drm,i2c_algo_bit, Live 0xffffffffa000b000
video 18932 1 i915, Live 0xffffffffa0000000

[8.4.]

0000-0cf7 : PCI Bus 0000:00
  0000-001f : dma1
  0020-0021 : pic1
  0040-0043 : timer0
  0050-0053 : timer1
  0060-0060 : keyboard
  0064-0064 : keyboard
  0070-0077 : rtc0
  0080-008f : dma page reg
  00a0-00a1 : pic2
  00c0-00df : dma2
  00f0-00ff : fpu
  0200-020f : pnp 00:04
  0290-029f : pnp 00:07
  03c0-03df : vga+
  0400-0453 : pnp 00:04
    0400-0403 : ACPI PM1a_EVT_BLK
    0404-0405 : ACPI PM1a_CNT_BLK
    0408-040b : ACPI PM_TMR
    0410-0415 : ACPI CPU throttle
    0420-042f : ACPI GPE0_BLK
    0450-0450 : ACPI PM2_CNT_BLK
  0454-0457 : pnp 00:06
  0458-047f : pnp 00:04
  04d0-04d1 : pnp 00:08
  0500-057f : pnp 00:04
  0680-069f : pnp 00:04
0cf8-0cff : PCI conf1
0d00-ffff : PCI Bus 0000:00
  164e-164f : pnp 00:04
  2000-2fff : PCI Bus 0000:02
  d000-dfff : PCI Bus 0000:05
    d000-d0ff : 0000:05:00.0
      d000-d0ff : r8169
  e000-efff : PCI Bus 0000:01
    e000-e0ff : 0000:01:00.0
  f000-f03f : 0000:00:02.0
  f040-f05f : 0000:00:1f.3
  f060-f07f : 0000:00:1f.2
    f060-f07f : ahci
  f080-f083 : 0000:00:1f.2
    f080-f083 : ahci
  f090-f097 : 0000:00:1f.2
    f090-f097 : ahci
  f0a0-f0a3 : 0000:00:1f.2
    f0a0-f0a3 : ahci
  f0b0-f0b7 : 0000:00:1f.2
    f0b0-f0b7 : ahci
  ffff-ffff : pnp 00:04
    ffff-ffff : pnp 00:04




00000000-0000ffff : reserved
00010000-0009d7ff : System RAM
0009d800-0009ffff : reserved
000a0000-000bffff : PCI Bus 0000:00
000c0000-000ce3ff : Video ROM
000d0000-000d3fff : PCI Bus 0000:00
000d4000-000d7fff : PCI Bus 0000:00
000d8000-000dbfff : PCI Bus 0000:00
000dc000-000dffff : PCI Bus 0000:00
000e0000-000fffff : reserved
  000e0000-000e3fff : PCI Bus 0000:00
  000e4000-000e7fff : PCI Bus 0000:00
  000e8000-000ebfff : PCI Bus 0000:00
  000f0000-000fffff : System ROM
00100000-1fffffff : System RAM
  01000000-0161d5ef : Kernel code
  0161d5f0-01cdf67f : Kernel data
  01de6000-01f9afff : Kernel bss
20000000-201fffff : reserved
  20000000-201fffff : pnp 00:0b
20200000-3fffffff : System RAM
40000000-401fffff : reserved
  40000000-401fffff : pnp 00:0b
40200000-adbc7fff : System RAM
adbc8000-ae13ffff : reserved
ae140000-ae3bffff : ACPI Non-volatile Storage
ae3c0000-ae3c4fff : ACPI Tables
ae3c5000-ae407fff : ACPI Non-volatile Storage
ae408000-aecc3fff : System RAM
aecc4000-aeff1fff : reserved
aeff2000-aeffffff : System RAM
af000000-af7fffff : RAM buffer
af800000-cf9fffff : reserved
cfa00000-feafffff : PCI Bus 0000:00
  cfa00000-cfa00fff : pnp 00:0a
  cfb00000-cfcfffff : PCI Bus 0000:02
  cfd00000-cfefffff : PCI Bus 0000:02
  d0000000-dfffffff : 0000:00:02.0
  e0000000-efffffff : PCI Bus 0000:01
    e0000000-efffffff : 0000:01:00.0
  f0000000-f00fffff : PCI Bus 0000:05
    f0000000-f0003fff : 0000:05:00.0
      f0000000-f0003fff : r8169
    f0004000-f0004fff : 0000:05:00.0
      f0004000-f0004fff : r8169
  f7800000-f7bfffff : 0000:00:02.0
  f7c00000-f7cfffff : PCI Bus 0000:06
    f7c00000-f7c07fff : 0000:06:00.0
      f7c00000-f7c07fff : xhci_hcd
  f7d00000-f7dfffff : PCI Bus 0000:01
    f7d00000-f7d1ffff : 0000:01:00.0
    f7d20000-f7d3ffff : 0000:01:00.0
    f7d40000-f7d43fff : 0000:01:00.1
      f7d40000-f7d43fff : ICH HD audio
  f7e00000-f7e03fff : 0000:00:1b.0
    f7e00000-f7e03fff : ICH HD audio
  f7e05000-f7e050ff : 0000:00:1f.3
  f7e06000-f7e067ff : 0000:00:1f.2
    f7e06000-f7e067ff : ahci
  f7e07000-f7e073ff : 0000:00:1d.0
    f7e07000-f7e073ff : ehci_hcd
  f7e08000-f7e083ff : 0000:00:1a.0
    f7e08000-f7e083ff : ehci_hcd
  f7e0b000-f7e0b00f : 0000:00:16.0
  f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f]
    f8000000-fbffffff : reserved
      f8000000-fbffffff : pnp 00:0a
fec00000-fec00fff : reserved
  fec00000-fec003ff : IOAPIC 0
fed00000-fed03fff : reserved
fed10000-fed17fff : pnp 00:0a
fed18000-fed18fff : pnp 00:0a
fed19000-fed19fff : pnp 00:0a
fed1c000-fed1ffff : reserved
  fed1c000-fed1ffff : pnp 00:0a
fed20000-fed3ffff : pnp 00:0a
fed40000-fed44fff : pnp 00:01
fed45000-fed8ffff : pnp 00:0a
fed90000-fed93fff : pnp 00:0a
fee00000-fee00fff : Local APIC
  fee00000-fee00fff : reserved
ff000000-ffffffff : reserved
  ff000000-ffffffff : pnp 00:0a
100000000-22f5fffff : System RAM
22f600000-22fffffff : RAM buffer

[8.5.]
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller (rev 09)
    Subsystem: ASRock Incorporation Device 0108
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
    Latency: 0
    Capabilities: [e0] Vendor Specific Information: Len=0c <?>
    Kernel driver in use: agpgart-intel

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) (prog-if 00 [Normal decode])
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
    I/O behind bridge: 0000e000-0000efff
    Memory behind bridge: f7d00000-f7dfffff
    Prefetchable memory behind bridge: 00000000e0000000-00000000efffffff
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
    BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [88] Subsystem: ASRock Incorporation Device 0101
    Capabilities: [80] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Address: fee002b8  Data: 0000
    Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
        DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
            ExtTag- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
        LnkCap: Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
            ClockPM- Surprise- LLActRep- BwNot+
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
        SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
            Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
        SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
            Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
        SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
            Changed: MRL- PresDet- LinkState-
        RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
        RootCap: CRSVisible-
        RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        DevCap2: Completion Timeout: Not Supported, TimeoutDis- ARIFwd-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB
    Capabilities: [100 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:        Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:       Enable+ ID=0 ArbSelect=Fixed TC/VC=01
            Status:     NegoPending- InProgress-
    Capabilities: [140 v1] Root Complex Link
        Desc:   PortNumber=02 ComponentID=01 EltType=Config
        Link0:  Desc:   TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
            Addr:       00000000fed19000
    Kernel driver in use: pcieport
    Kernel modules: shpchp

00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 Processor Family Integrated Graphics Controller (rev 09) (prog-if 00 [VGA controller])
    Subsystem: ASRock Incorporation Device 010a
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 48
    Region 0: Memory at f7800000 (64-bit, non-prefetchable) [size=4M]
    Region 2: Memory at d0000000 (64-bit, prefetchable) [size=256M]
    Region 4: I/O ports at f000 [size=64]
    Expansion ROM at <unassigned> [disabled]
    Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Address: fee00018  Data: 0000
    Capabilities: [d0] Power Management version 2
        Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [a4] PCI Advanced Features
        AFCap: TP+ FLR+
        AFCtrl: FLR-
        AFStatus: TP-
    Kernel driver in use: i915
    Kernel modules: i915

00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
    Subsystem: ASRock Incorporation Device 1c3a
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 11
    Region 0: Memory at f7e0b000 (64-bit, non-prefetchable) [size=16]
    Capabilities: [50] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [8c] MSI: Enable- Count=1/1 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000

00:1a.0 USB Controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05) (prog-if 20 [EHCI])
    Subsystem: ASRock Incorporation Device 1c2d
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 16
    Region 0: Memory at f7e08000 (32-bit, non-prefetchable) [size=1K]
    Capabilities: [50] Power Management version 2
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [58] Debug port: BAR=1 offset=00a0
    Capabilities: [98] PCI Advanced Features
        AFCap: TP+ FLR+
        AFCtrl: FLR-
        AFStatus: TP-
    Kernel driver in use: ehci_hcd

00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
    Subsystem: ASRock Incorporation Device 1892
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 50
    Region 0: Memory at f7e00000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [50] Power Management version 2
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Address: 00000000fee003d8  Data: 0000
    Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00
        DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
            ExtTag- RBE- FLReset+
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us
            ClockPM- Surprise- LLActRep- BwNot-
        LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
    Capabilities: [100 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:        Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:       Enable+ ID=0 ArbSelect=Fixed TC/VC=01
            Status:     NegoPending- InProgress-
        VC1:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:        Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:       Enable+ ID=1 ArbSelect=Fixed TC/VC=22
            Status:     NegoPending- InProgress-
    Capabilities: [130 v1] Root Complex Link
        Desc:   PortNumber=0f ComponentID=00 EltType=Config
        Link0:  Desc:   TargetPort=00 TargetComponent=00 AssocRCRB- LinkType=MemMapped LinkValid+
            Addr:       00000000fed1c000
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd-hda-intel

00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5) (prog-if 00 [Normal decode])
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
    I/O behind bridge: 00002000-00002fff
    Memory behind bridge: cfb00000-cfcfffff
    Prefetchable memory behind bridge: 00000000cfd00000-00000000cfefffff
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
    BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
        DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
            ExtTag- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <4us
            ClockPM- Surprise- LLActRep+ BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
            Slot #0, PowerLimit 0.000W; Interlock- NoCompl+
        SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
            Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
        SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
            Changed: MRL- PresDet- LinkState-
        RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
        RootCap: CRSVisible-
        RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        DevCap2: Completion Timeout: Range BC, TimeoutDis+ ARIFwd-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -3.5dB
    Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Address: fee002d8  Data: 0000
    Capabilities: [90] Subsystem: ASRock Incorporation Device 1c10
    Capabilities: [a0] Power Management version 2
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Kernel driver in use: pcieport
    Kernel modules: shpchp

00:1c.4 PCI bridge: Intel Corporation 82801 PCI Bridge (rev b5) (prog-if 01 [Subtractive decode])
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Bus: primary=00, secondary=03, subordinate=04, sec-latency=0
    I/O behind bridge: 0000f000-00000fff
    Memory behind bridge: fff00000-000fffff
    Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
    BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
        DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
            ExtTag- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #5, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <4us
            ClockPM- Surprise- LLActRep+ BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
        SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
            Slot #4, PowerLimit 10.000W; Interlock- NoCompl+
        SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
            Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
        SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
            Changed: MRL- PresDet- LinkState-
        RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
        RootCap: CRSVisible-
        RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        DevCap2: Completion Timeout: Range BC, TimeoutDis+ ARIFwd-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -3.5dB
    Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
        Address: 00000000  Data: 0000
    Capabilities: [90] Subsystem: ASRock Incorporation Device 244e
    Capabilities: [a0] Power Management version 2
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b5) (prog-if 00 [Normal decode])
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
    I/O behind bridge: 0000d000-0000dfff
    Memory behind bridge: fff00000-000fffff
    Prefetchable memory behind bridge: 00000000f0000000-00000000f00fffff
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
    BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
        DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
            ExtTag- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <4us
            ClockPM- Surprise- LLActRep+ BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
        SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
            Slot #5, PowerLimit 10.000W; Interlock- NoCompl+
        SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
            Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
        SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
            Changed: MRL- PresDet- LinkState-
        RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
        RootCap: CRSVisible-
        RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        DevCap2: Completion Timeout: Range BC, TimeoutDis+ ARIFwd-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -3.5dB
    Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Address: fee002f8  Data: 0000
    Capabilities: [90] Subsystem: ASRock Incorporation Device 1c1a
    Capabilities: [a0] Power Management version 2
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Kernel driver in use: pcieport
    Kernel modules: shpchp

00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5) (prog-if 00 [Normal decode])
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
    I/O behind bridge: 0000f000-00000fff
    Memory behind bridge: f7c00000-f7cfffff
    Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
    BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
        DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
            ExtTag- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #7, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <4us
            ClockPM- Surprise- LLActRep+ BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
        SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
            Slot #6, PowerLimit 10.000W; Interlock- NoCompl+
        SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
            Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
        SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
            Changed: MRL- PresDet- LinkState-
        RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
        RootCap: CRSVisible-
        RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        DevCap2: Completion Timeout: Range BC, TimeoutDis+ ARIFwd-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB
    Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Address: fee00318  Data: 0000
    Capabilities: [90] Subsystem: ASRock Incorporation Device 1c1c
    Capabilities: [a0] Power Management version 2
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Kernel driver in use: pcieport
    Kernel modules: shpchp

00:1d.0 USB Controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05) (prog-if 20 [EHCI])
    Subsystem: ASRock Incorporation Device 1c26
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 23
    Region 0: Memory at f7e07000 (32-bit, non-prefetchable) [size=1K]
    Capabilities: [50] Power Management version 2
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [58] Debug port: BAR=1 offset=00a0
    Capabilities: [98] PCI Advanced Features
        AFCap: TP+ FLR+
        AFCtrl: FLR-
        AFStatus: TP-
    Kernel driver in use: ehci_hcd

00:1f.0 ISA bridge: Intel Corporation Z68 Express Chipset Family LPC Controller (rev 05)
    Subsystem: ASRock Incorporation Device 1c44
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Capabilities: [e0] Vendor Specific Information: Len=0c <?>
    Kernel modules: lpc_ich

00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port SATA AHCI Controller (rev 05) (prog-if 01 [AHCI 1.0])
    Subsystem: ASRock Incorporation Device 1c02
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin B routed to IRQ 46
    Region 0: I/O ports at f0b0 [size=8]
    Region 1: I/O ports at f0a0 [size=4]
    Region 2: I/O ports at f090 [size=8]
    Region 3: I/O ports at f080 [size=4]
    Region 4: I/O ports at f060 [size=32]
    Region 5: Memory at f7e06000 (32-bit, non-prefetchable) [size=2K]
    Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Address: fee00358  Data: 0000
    Capabilities: [70] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004
    Capabilities: [b0] PCI Advanced Features
        AFCap: TP+ FLR+
        AFCtrl: FLR-
        AFStatus: TP-
    Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
    Subsystem: ASRock Incorporation Device 1c22
    Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Interrupt: pin C routed to IRQ 18
    Region 0: Memory at f7e05000 (64-bit, non-prefetchable) [size=256]
    Region 4: I/O ports at f040 [size=32]
    Kernel modules: i2c-i801

01:00.0 VGA compatible controller: ATI Technologies Inc NI Turks [AMD Radeon HD 6500] (prog-if 00 [VGA controller])
    Subsystem: PC Partner Limited Device e193
    Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Interrupt: pin A routed to IRQ 11
    Region 0: Memory at e0000000 (64-bit, prefetchable) [disabled] [size=256M]
    Region 2: Memory at f7d20000 (64-bit, non-prefetchable) [disabled] [size=128K]
    Region 4: I/O ports at e000 [disabled] [size=256]
    Expansion ROM at f7d00000 [disabled] [size=128K]
    Capabilities: [50] Power Management version 3
        Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
        DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 128 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
        LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
            ClockPM- Surprise- LLActRep- BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
        LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB
    Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150 v1] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
    Kernel modules: radeon

01:00.1 Audio device: ATI Technologies Inc Device aa90
    Subsystem: PC Partner Limited Device aa90
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin B routed to IRQ 51
    Region 0: Memory at f7d40000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [50] Power Management version 3
        Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
        DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 128 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
        LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
            ClockPM- Surprise- LLActRep- BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
        LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Address: 00000000fee003f8  Data: 0000
    Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150 v1] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd-hda-intel

03:00.0 PCI bridge: ASMedia Technology Inc. Device 1080 (rev 01) (prog-if 01 [Subtractive decode])
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Bus: primary=03, secondary=04, subordinate=04, sec-latency=32
    I/O behind bridge: 0000f000-00000fff
    Memory behind bridge: fff00000-000fffff
    Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
    Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
    BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [c0] Subsystem: ASRock Incorporation Device 1080

05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)
    Subsystem: ASRock Incorporation Motherboard (one of many)
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 49
    Region 0: I/O ports at d000 [size=256]
    Region 2: Memory at f0004000 (64-bit, prefetchable) [size=4K]
    Region 4: Memory at f0000000 (64-bit, prefetchable) [size=16K]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Address: 00000000fee00398  Data: 0000
    Capabilities: [70] Express (v2) Endpoint, MSI 01
        DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 4096 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 unlimited, L1 <64us
            ClockPM+ Surprise- LLActRep- BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
        LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB
    Capabilities: [b0] MSI-X: Enable- Count=4 Masked-
        Vector table: BAR=4 offset=00000000
        PBA: BAR=4 offset=00000800
    Capabilities: [d0] Vital Product Data
        No end tag found
    Capabilities: [100 v1] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn-
    Capabilities: [140 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:        Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:       Enable+ ID=0 ArbSelect=Fixed TC/VC=01
            Status:     NegoPending- InProgress-
    Capabilities: [160 v1] Device Serial Number 01-00-00-00-68-4c-e0-00
    Kernel driver in use: r8169
    Kernel modules: r8169

06:00.0 USB Controller: Device 1b6f:7023 (rev 01) (prog-if 30 [XHCI])
    Subsystem: ASRock Incorporation Device 7023
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 47
    Region 0: Memory at f7c00000 (64-bit, non-prefetchable) [size=32K]
    Capabilities: [50] Power Management version 3
        Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [70] MSI: Enable+ Count=1/4 Maskable+ 64bit+
        Address: 00000000fee00378  Data: 0000
        Masking: 0000000e  Pending: 00000000
    Capabilities: [a0] Express (v2) Endpoint, MSI 00
        DevCap: MaxPayload 1024 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
            MaxPayload 128 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <64us
            ClockPM+ Surprise- LLActRep- BwNot-
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Not Supported, TimeoutDis-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
        LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB
    Capabilities: [100 v1] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO+ CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout+ NonFatalErr+
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn-
    Capabilities: [190 v1] Device Serial Number 01-01-01-01-01-01-01-01
    Kernel driver in use: xhci_hcd
Comment 1 Reuben Farrelly 2012-08-03 12:06:29 UTC
+1 here.  This on a Genuine Intel DZ77GA-70K Mainboard:

IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
------------[ cut here ]------------
WARNING: at drivers/pci/search.c:44 pci_find_upstream_pcie_bridge+0x50/0x70()
Hardware name:         
Modules linked in:
Pid: 1, comm: swapper/0 Not tainted 3.5.0-gentoo #1
Call Trace:
 [<ffffffff8102c65a>] warn_slowpath_common+0x7a/0xb0
 [<ffffffff8102c6a5>] warn_slowpath_null+0x15/0x20
 [<ffffffff8124b6e0>] pci_find_upstream_pcie_bridge+0x50/0x70
 [<ffffffff813c3d6d>] intel_iommu_device_group+0x7d/0x100
 [<ffffffff813bedd0>] ? bus_set_iommu+0x50/0x50
 [<ffffffff813bedfb>] add_iommu_group+0x2b/0x60
 [<ffffffff812d88ad>] bus_for_each_dev+0x4d/0x90
 [<ffffffff813bedbb>] bus_set_iommu+0x3b/0x50
 [<ffffffff816accb7>] intel_iommu_init+0xaf0/0xc16
 [<ffffffff81023b73>] ? free_init_pages+0x123/0x1b0
 [<ffffffff81681ad1>] ? md_run_setup+0x9c/0x9c
 [<ffffffff81685f2d>] ? memblock_find_dma_reserve+0x13d/0x13d
 [<ffffffff81685f40>] pci_iommu_init+0x13/0x3e
 [<ffffffff8100029a>] do_one_initcall+0x11a/0x160
 [<ffffffff81680cba>] kernel_init+0x135/0x1b9
 [<ffffffff816805af>] ? do_early_param+0x8c/0x8c
 [<ffffffff814ae0d4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81680b85>] ? start_kernel+0x377/0x377
 [<ffffffff814ae0d0>] ? gs_change+0xb/0xb
---[ end trace a307b3ed40206b48 ]---
microcode: CPU0 sig=0x206a7, pf=0x2, revision=0x26

tornado ~ # lspci -t
-[0000:00]-+-00.0
           +-01.0-[01]--+-00.0
           |            \-00.1
           +-01.1-[02]--+-00.0
           |            \-00.1
           +-16.0
           +-1a.0
           +-1c.0-[03]--
           +-1c.1-[04]----00.0
           +-1c.3-[05-06]----00.0-[06]--
           +-1c.4-[07-08]----00.0-[08]----00.0
           +-1c.7-[09]----00.0
           +-1d.0
           +-1f.0
           +-1f.2
           \-1f.3
tornado ~ # 

tornado ~ # lspci -vvv
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
        Subsystem: Intel Corporation Device 2010
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
        Latency: 0
        Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00003000-00003fff
        Memory behind bridge: f0500000-f05fffff
        Prefetchable memory behind bridge: 00000000e0000000-00000000efffffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [88] Subsystem: Intel Corporation Device 2010
        Capabilities: [80] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4161
        Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #2, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <256ns, L1 <4us
                        ClockPM- Surprise- LLActRep- BwNot+
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet+ LinkState-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                        Status: NegoPending- InProgress-
        Capabilities: [140 v1] Root Complex Link
                Desc:   PortNumber=02 ComponentID=01 EltType=Config
                Link0:  Desc:   TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
                        Addr:   00000000fed19000
        Kernel driver in use: pcieport

00:01.1 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: f0200000-f04fffff
        Prefetchable memory behind bridge: 00000000f0800000-00000000f08fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [88] Subsystem: Intel Corporation Device 2010
        Capabilities: [80] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4169
        Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #3, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <256ns, L1 <4us
                        ClockPM- Surprise- LLActRep- BwNot+
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt+
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #2, PowerLimit 75.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet+ LinkState-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                        Status: NegoPending- InProgress-
        Capabilities: [140 v1] Root Complex Link
                Desc:   PortNumber=03 ComponentID=01 EltType=Config
                Link0:  Desc:   TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
                        Addr:   00000000fed19000
        Kernel driver in use: pcieport

00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
        Subsystem: Intel Corporation Device 4742
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 48
        Region 0: Memory at f0700000 (64-bit, non-prefetchable) [size=16]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [8c] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000feeff00c  Data: 4199
        Kernel driver in use: mei

00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05) (prog-if 20 [EHCI])
        Subsystem: Intel Corporation Device 4742
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at f0701000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [50] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Debug port: BAR=1 offset=00a0
        Capabilities: [98] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: ehci_hcd

00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5) (prog-if 00 [Normal decode])
        Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #0, PowerLimit 10.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
                        Changed: MRL- PresDet- LinkState-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4171
        Capabilities: [90] Subsystem: Intel Corporation Device 4742
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: pcieport

00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b5) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: f0100000-f01fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot-), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #2, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4179
        Capabilities: [90] Subsystem: Intel Corporation Device 4742
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: pcieport

00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 4 (rev b5) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=05, subordinate=06, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot-), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #4, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4181
        Capabilities: [90] Subsystem: Intel Corporation Device 4742
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: pcieport

00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=07, subordinate=08, sec-latency=0
        I/O behind bridge: 00002000-00002fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #5, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #7, PowerLimit 10.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4189
        Capabilities: [90] Subsystem: Intel Corporation Device 4742
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: pcieport

00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=09, subordinate=09, sec-latency=0
        I/O behind bridge: 00001000-00001fff
        Memory behind bridge: f0000000-f00fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot-), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #8, Speed 5GT/s, Width x1, ASPM unknown, Latency L0 <512ns, L1 <4us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4191
        Capabilities: [90] Subsystem: Intel Corporation Device 4742
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: pcieport

00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05) (prog-if 20 [EHCI])
        Subsystem: Intel Corporation Device 4742
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 23
        Region 0: Memory at f0702000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [50] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Debug port: BAR=1 offset=00a0
        Capabilities: [98] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: ehci_hcd

00:1f.0 ISA bridge: Intel Corporation P67 Express Chipset Family LPC Controller (rev 05)
        Subsystem: Intel Corporation Device 4742
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05) (prog-if 01 [AHCI 1.0])
        Subsystem: Intel Corporation Device 4742
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin B routed to IRQ 49
        Region 0: I/O ports at 4048 [size=8]
        Region 1: I/O ports at 4054 [size=4]
        Region 2: I/O ports at 4040 [size=8]
        Region 3: I/O ports at 4050 [size=4]
        Region 4: I/O ports at 4020 [size=32]
        Region 5: Memory at f0601000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 41a1
        Capabilities: [70] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004
        Capabilities: [b0] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
        Subsystem: Intel Corporation Device 4742
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin C routed to IRQ 10
        Region 0: Memory at f0602000 (64-bit, non-prefetchable) [size=256]
        Region 4: I/O ports at 4000 [size=32]

01:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI RV710 [Radeon HD 4350] (prog-if 00 [VGA controller])
        Subsystem: Giga-byte Technology Device 21ac
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 11
        Region 0: Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Region 2: Memory at f0500000 (64-bit, non-prefetchable) [size=64K]
        Region 4: I/O ports at 3000 [size=256]
        Expansion ROM at f0520000 [disabled] [size=128K]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>

01:00.1 Audio device: Advanced Micro Devices [AMD] nee ATI RV710/730 HDMI Audio [Radeon HD 4000 series]
        Subsystem: Giga-byte Technology Device aa38
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin B routed to IRQ 9
        Region 0: Memory at f0510000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>

02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Ethernet Server Adapter I350-T2
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 17
        Region 0: Memory at f0300000 (32-bit, non-prefetchable) [size=1M]
        Region 3: Memory at f0404000 (32-bit, non-prefetchable) [size=16K]
        Expansion ROM at f0800000 [disabled] [size=512K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
                Vector table: BAR=3 offset=00000000
                PBA: BAR=3 offset=00002000
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
                DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <32us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [140 v1] Device Serial Number a0-36-9f-ff-ff-00-f4-aa
        Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
                ARICap: MFVC- ACS-, Next Function: 1
                ARICtl: MFVC- ACS-, Function Group: 0
        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
                IOVCap: Migration-, Interrupt Message Number: 000
                IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
                IOVSta: Migration-
                Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 00
                VF offset: 384, stride: 4, Device ID: 1520
                Supported Page Size: 00000553, System Page Size: 00000001
                Region 0: Memory at f0408000 (32-bit, non-prefetchable)
                Region 3: Memory at f0428000 (32-bit, non-prefetchable)
                VF Migration: offset: 00000000, BIR: 0
        Capabilities: [1a0 v1] Transaction Processing Hints
                Device specific mode supported
                Steering table in TPH capability structure
        Capabilities: [1c0 v1] Latency Tolerance Reporting
                Max snoop latency: 0ns
                Max no snoop latency: 0ns
        Capabilities: [1d0 v1] Access Control Services
                ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
                ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
        Kernel driver in use: igb

02:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Ethernet Server Adapter I350-T2
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin B routed to IRQ 18
        Region 0: Memory at f0200000 (32-bit, non-prefetchable) [size=1M]
        Region 3: Memory at f0400000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
                Vector table: BAR=3 offset=00000000
                PBA: BAR=3 offset=00002000
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
                DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <32us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [140 v1] Device Serial Number a0-36-9f-ff-ff-00-f4-aa
        Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
                ARICap: MFVC- ACS-, Next Function: 0
                ARICtl: MFVC- ACS-, Function Group: 0
        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
                IOVCap: Migration-, Interrupt Message Number: 000
                IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
                IOVSta: Migration-
                Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 01
                VF offset: 384, stride: 4, Device ID: 1520
                Supported Page Size: 00000553, System Page Size: 00000001
                Region 0: Memory at f0448000 (32-bit, non-prefetchable)
                Region 3: Memory at f0468000 (32-bit, non-prefetchable)
                VF Migration: offset: 00000000, BIR: 0
        Capabilities: [1a0 v1] Transaction Processing Hints
                Device specific mode supported
                Steering table in TPH capability structure
        Capabilities: [1d0 v1] Access Control Services
                ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
                ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
        Kernel driver in use: igb

04:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03) (prog-if 30 [XHCI])
        Subsystem: Intel Corporation Device 4742
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 17
        Region 0: Memory at f0100000 (64-bit, non-prefetchable) [size=8K]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [90] MSI-X: Enable+ Count=8 Masked-
                Vector table: BAR=0 offset=00001000
                PBA: BAR=0 offset=00001080
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <4us, L1 unlimited
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR+, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
        Capabilities: [140 v1] Device Serial Number ff-ff-ff-ff-ff-ff-ff-ff
        Capabilities: [150 v1] Latency Tolerance Reporting
                Max snoop latency: 0ns
                Max no snoop latency: 0ns
        Kernel driver in use: xhci_hcd

05:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8113 (prog-if 01 [Subtractive decode])
        Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=05, secondary=06, subordinate=06, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [a0] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-

07:00.0 PCI bridge: Texas Instruments XIO2000(A)/XIO2200A PCI Express-to-PCI Bridge (rev 03) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=07, secondary=08, subordinate=08, sec-latency=0
        I/O behind bridge: 00002000-00002fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B+
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [50] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
                Bridge: PM- B3+
        Capabilities: [60] MSI: Enable- Count=1/16 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [80] Subsystem: Gammagraphx, Inc. (or missing ID) Device 0000
        Capabilities: [90] Express (v1) PCI/PCI-X Bridge, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <4us, L1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ BrConfRtry-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <16us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-

08:00.0 Serial controller: NetMos Technology PCI 9835 Multi-I/O Controller (rev 01) (prog-if 02 [16550])
        Subsystem: LSI Logic / Symbios Logic 2S (16C550 UART)
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 16
        Region 0: I/O ports at 2030 [size=8]
        Region 1: I/O ports at 2028 [size=8]
        Region 2: I/O ports at 2020 [size=8]
        Region 3: I/O ports at 2018 [size=8]
        Region 4: I/O ports at 2010 [size=8]
        Region 5: I/O ports at 2000 [size=16]
        Kernel driver in use: serial

09:00.0 IDE interface: Marvell Technology Group Ltd. 88SE6121 SATA II / PATA Controller (rev b2) (prog-if 8f [Master SecP SecO PriP PriO])
        Subsystem: Intel Corporation Device 6111
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 19
        Region 0: I/O ports at 1018 [size=8]
        Region 1: I/O ports at 1024 [size=4]
        Region 2: I/O ports at 1010 [size=8]
        Region 3: I/O ports at 1020 [size=4]
        Region 4: I/O ports at 1000 [size=16]
        Region 5: Memory at f0000000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [48] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [e0] Express (v1) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr+ NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 1f, GenCap- CGenEn- ChkCap- ChkEn-
        Kernel driver in use: ahci

tornado ~ #
Comment 2 Bjorn Helgaas 2012-08-06 22:37:02 UTC
Created attachment 76931 [details]
debug patch

This won't fix anything, but will show us which device we're concerned about.  If you can test this, please attach the complete dmesg log.
Comment 3 Reuben Farrelly 2012-08-07 01:38:07 UTC
Ok on my box:

Unpacking initramfs...
Freeing initrd memory: 9036k freed
DMAR: No ATSR found
IOMMU 0 0xfed91000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xdf6a3000 - 0xdf6a3fff]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xdf69e000 - 0xdf69efff]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
pci 0000:05:00.0: expected upstream PCIe bridge, but 0000:00:1c.3 is type 0x4
------------[ cut here ]------------
WARNING: at drivers/pci/search.c:47 pci_find_upstream_pcie_bridge+0x95/0xb0()
Hardware name:
Modules linked in:
Pid: 1, comm: swapper/0 Not tainted 3.5.0-gentoo #2
Call Trace:
 [<ffffffff8102c65a>] warn_slowpath_common+0x7a/0xb0
 [<ffffffff8102c6a5>] warn_slowpath_null+0x15/0x20
 [<ffffffff8124b725>] pci_find_upstream_pcie_bridge+0x95/0xb0
 [<ffffffff813c3dad>] intel_iommu_device_group+0x7d/0x100
 [<ffffffff813bee10>] ? bus_set_iommu+0x50/0x50
 [<ffffffff813bee3b>] add_iommu_group+0x2b/0x60
 [<ffffffff812d88ed>] bus_for_each_dev+0x4d/0x90
 [<ffffffff813bedfb>] bus_set_iommu+0x3b/0x50
 [<ffffffff816accb7>] intel_iommu_init+0xaf0/0xc16
 [<ffffffff81023b73>] ? free_init_pages+0x123/0x1b0
 [<ffffffff81681ad1>] ? md_run_setup+0x9c/0x9c
 [<ffffffff81685f2d>] ? memblock_find_dma_reserve+0x13d/0x13d
 [<ffffffff81685f40>] pci_iommu_init+0x13/0x3e
 [<ffffffff8100029a>] do_one_initcall+0x11a/0x160
 [<ffffffff81680cba>] kernel_init+0x135/0x1b9
 [<ffffffff816805af>] ? do_early_param+0x8c/0x8c
 [<ffffffff814ae114>] kernel_thread_helper+0x4/0x10
 [<ffffffff81680b85>] ? start_kernel+0x377/0x377
 [<ffffffff814ae110>] ? gs_change+0xb/0xb
---[ end trace a307b3ed40206b48 ]---
microcode: CPU0 sig=0x206a7, pf=0x2, revision=0x26
microcode: CPU1 sig=0x206a7, pf=0x2, revision=0x26
microcode: CPU2 sig=0x206a7, pf=0x2, revision=0x26
microcode: CPU3 sig=0x206a7, pf=0x2, revision=0x26
microcode: CPU4 sig=0x206a7, pf=0x2, revision=0x26
microcode: CPU5 sig=0x206a7, pf=0x2, revision=0x26


Which I guess indicates this is the problem child:

05:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8113 (prog-if 01 [Subtractive decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=05, secondary=06, subordinate=06, sec-latency=0
        Capabilities: [a0] Power Management version 3

I'm not too sure what it is though.  Not in front of the box now but the vendor name doesn't ring any bells.

[Incidentally I made a mistake in my earlier comment, this wasn't for a DZ77GA-70K board but an Intel DP67BG board]
Comment 4 John Wehin 2012-08-07 19:53:18 UTC
[    0.317700] Unpacking initramfs...
[    0.527283] Freeing initrd memory: 16628k freed
[    0.528933] DMAR: No ATSR found
[    0.528958] IOMMU 1 0xfed91000: using Queued invalidation
[    0.528960] IOMMU: Setting RMRR:
[    0.528968] IOMMU: Setting identity map for device 0000:00:1d.0 [0xade62000 - 0xade8cfff]
[    0.528987] IOMMU: Setting identity map for device 0000:00:1a.0 [0xade62000 - 0xade8cfff]
[    0.528996] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.529003] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.529095] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.529112] pci 0000:03:00.0: expected upstream PCIe bridge, but 0000:00:1c.4 is type 0x4
[    0.529113] ------------[ cut here ]------------
[    0.529118] WARNING: at drivers/pci/search.c:47 pci_find_upstream_pcie_bridge+0x9e/0xb0()
[    0.529119] Hardware name: To Be Filled By O.E.M.
[    0.529120] Modules linked in:
[    0.529122] Pid: 1, comm: swapper/0 Not tainted 3.5.0-2.f2f.fc16.x86_64 #1
[    0.529123] Call Trace:
[    0.529128]  [<ffffffff8105928f>] warn_slowpath_common+0x7f/0xc0
[    0.529130]  [<ffffffff810592ea>] warn_slowpath_null+0x1a/0x20
[    0.529132]  [<ffffffff812f54ae>] pci_find_upstream_pcie_bridge+0x9e/0xb0
[    0.529135]  [<ffffffff814d2bcc>] intel_iommu_device_group+0x7c/0x100
[    0.529138]  [<ffffffff814c857a>] add_iommu_group+0x3a/0x60
[    0.529139]  [<ffffffff814c8540>] ? bus_set_iommu+0x50/0x50
[    0.529142]  [<ffffffff813aed56>] bus_for_each_dev+0x56/0x90
[    0.529144]  [<ffffffff814c8530>] bus_set_iommu+0x40/0x50
[    0.529147]  [<ffffffff81d343e6>] intel_iommu_init+0xa9b/0xb2a
[    0.529149]  [<ffffffff81042690>] ? free_init_pages+0x130/0x1c0
[    0.529152]  [<ffffffff81cf6b3c>] ? md_run_setup+0x8b/0x8b
[    0.529154]  [<ffffffff81cfd002>] ? memblock_find_dma_reserve+0x13d/0x13d
[    0.529156]  [<ffffffff81cfd015>] pci_iommu_init+0x13/0x3e
[    0.529159]  [<ffffffff8100203f>] do_one_initcall+0x3f/0x170
[    0.529161]  [<ffffffff81cf5d7c>] kernel_init+0x14e/0x1d2
[    0.529163]  [<ffffffff81cf5610>] ? do_early_param+0x86/0x86
[    0.529167]  [<ffffffff81608424>] kernel_thread_helper+0x4/0x10
[    0.529169]  [<ffffffff81cf5c2e>] ? start_kernel+0x3c4/0x3c4
[    0.529171]  [<ffffffff81608420>] ? gs_change+0x13/0x13
[    0.529175] ---[ end trace a307b3ed40206b48 ]---
[    0.532958] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni)
[    0.533147] audit: initializing netlink socket (disabled)
[    0.533155] type=2000 audit(1344383072.413:1): initialized
[    0.546986] HugeTLB registered 2 MB page size, pre-allocated 0 pages
Comment 5 Alessandro Vietta 2012-08-23 00:30:02 UTC
Same warning.
Motherboard Intel DH77KC, BIOS 0095 (6/8/2012).

Linux version 3.5.2-ck1 (root@sertan) (gcc version 4.6.3 (Gentoo 4.6.3 p1.3, pie-0.5.2) ) #9 SMP PREEMPT Wed Aug 22 23:29:15 CEST 2012
Command line: BOOT_IMAGE=/vmlinuz-3.5.2-ck1 root=/dev/sda3 ro rootfstype=ext4 mtrr_gran_size=32M mtrr_chunk_size=128M clocksource=hpet quiet

[    0.283069] DMAR: Host address width 36
[    0.283070] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.283077] IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[    0.283078] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.283082] IOMMU 1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[    0.283083] DMAR: RMRR base: 0x000000da260000 end: 0x000000da27ffff
[    0.283084] DMAR: RMRR base: 0x000000db800000 end: 0x000000df9fffff
[    0.283091] DMAR: No ATSR found
[    0.283107] IOMMU 0 0xfed90000: using Queued invalidation
[    0.283109] IOMMU 1 0xfed91000: using Queued invalidation
[    0.283110] IOMMU: Setting RMRR:
[    0.283116] IOMMU: Setting identity map for device 0000:00:02.0 [0xdb800000 - 0xdf9fffff]
[    0.283401] IOMMU: Setting identity map for device 0000:00:1d.0 [0xda260000 - 0xda27ffff]
[    0.283416] IOMMU: Setting identity map for device 0000:00:1a.0 [0xda260000 - 0xda27ffff]
[    0.283427] IOMMU: Setting identity map for device 0000:00:14.0 [0xda260000 - 0xda27ffff]
[    0.283435] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.283440] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.283873] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.283879] pci 0000:02:00.0: expected upstream PCIe bridge, but 0000:00:1c.7 is type 0x4
[    0.283880] ------------[ cut here ]------------
[    0.283883] WARNING: at drivers/pci/search.c:47 pci_find_upstream_pcie_bridge+0x9e/0xb0()
[    0.283884] Hardware name:         
[    0.283884] Modules linked in:
[    0.283886] Pid: 1, comm: BFS/0 Not tainted 3.5.2-ck1 #9
[    0.283887] Call Trace:
[    0.283891]  [<ffffffff8104a3eb>] ? warn_slowpath_common+0x7b/0xc0
[    0.283892]  [<ffffffff8129d24e>] ? pci_find_upstream_pcie_bridge+0x9e/0xb0
[    0.283895]  [<ffffffff8146b035>] ? intel_iommu_device_group+0x75/0x100
[    0.283897]  [<ffffffff81465943>] ? add_iommu_group+0x33/0x60
[    0.283898]  [<ffffffff81465910>] ? bus_set_iommu+0x40/0x40
[    0.283900]  [<ffffffff8138210d>] ? bus_for_each_dev+0x4d/0x80
[    0.283901]  [<ffffffff81465903>] ? bus_set_iommu+0x33/0x40
[    0.283903]  [<ffffffff818a125e>] ? intel_iommu_init+0x9fa/0xa81
[    0.283906]  [<ffffffff8106f0e0>] ? lg_local_unlock+0x20/0x40
[    0.283908]  [<ffffffff811066a5>] ? sys_mkdirat+0x75/0xd0
[    0.283910]  [<ffffffff8188083a>] ? memblock_find_dma_reserve+0x13e/0x13e
[    0.283912]  [<ffffffff81880848>] ? pci_iommu_init+0xe/0x37
[    0.283913]  [<ffffffff810001b9>] ? do_one_initcall+0x39/0x170
[    0.283915]  [<ffffffff8187db30>] ? kernel_init+0x132/0x1b6
[    0.283916]  [<ffffffff8187d460>] ? loglevel+0x34/0x34
[    0.283919]  [<ffffffff815ba554>] ? kernel_thread_helper+0x4/0x10
[    0.283920]  [<ffffffff8187d9fe>] ? start_kernel+0x2e3/0x2e3
[    0.283922]  [<ffffffff815ba550>] ? gs_change+0xb/0xb
[    0.283924] ---[ end trace 6b80ac37e558861f ]---
[    0.283926] pci 0000:03:00.0: expected upstream PCIe bridge, but 0000:00:1c.7 is type 0x4

Details (-vvv) of the problematic device:
00:1c.7 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 (rev c4) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=02, subordinate=03, sec-latency=0
        I/O behind bridge: 0000e000-0000efff
        Memory behind bridge: f7c00000-f7cfffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #8, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <16us
                        ClockPM- Surprise- LLActRep+ BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #7, PowerLimit 10.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 4161
        Capabilities: [90] Subsystem: Intel Corporation Device 2031
        Capabilities: [a0] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: pcieport

This motherboard has a PCI Express 3.0 x 16 port and two PCI Express 2.0 (x1 and x4).
00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)
00:1c.7 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 (rev c4)

-[0000:00]-+-00.0
           +-02.0
           +-14.0
           +-16.0
           +-19.0
           +-1a.0
           +-1b.0
           +-1c.0-[01]--
           +-1c.7-[02-03]----00.0-[03]----00.0
           +-1d.0
           +-1f.0
           +-1f.2
           \-1f.3
Comment 6 Alessandro Vietta 2012-08-23 01:43:09 UTC
Same warning with 3.6.0-rc3.
Motherboard Intel DH77KC, BIOS 0095 (6/8/2012).

Linux version 3.6.0-rc3 (root@sertan) (gcc version 4.6.3 (Gentoo 4.6.3 p1.3, pie-0.5.2) ) #1 SMP PREEMPT Thu Aug 23 03:31:14 CEST 2012
Command line: BOOT_IMAGE=/vmlinuz-3.6.0-rc3 root=/dev/sda3 ro rootfstype=ext4 mtrr_gran_size=32M mtrr_chunk_size=128M clocksource=hpet quiet

[    0.310144] dmar: Host address width 36
[    0.310145] dmar: DRHD base: 0x000000fed90000 flags: 0x0
[    0.310153] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[    0.310154] dmar: DRHD base: 0x000000fed91000 flags: 0x1
[    0.310158] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[    0.310159] dmar: RMRR base: 0x000000da260000 end: 0x000000da27ffff
[    0.310160] dmar: RMRR base: 0x000000db800000 end: 0x000000df9fffff
[    0.310166] DMAR: No ATSR found
[    0.310182] IOMMU 0 0xfed90000: using Queued invalidation
[    0.310184] IOMMU 1 0xfed91000: using Queued invalidation
[    0.310185] IOMMU: Setting RMRR:
[    0.310191] IOMMU: Setting identity map for device 0000:00:02.0 [0xdb800000 - 0xdf9fffff]
[    0.310471] IOMMU: Setting identity map for device 0000:00:1d.0 [0xda260000 - 0xda27ffff]
[    0.310486] IOMMU: Setting identity map for device 0000:00:1a.0 [0xda260000 - 0xda27ffff]
[    0.310498] IOMMU: Setting identity map for device 0000:00:14.0 [0xda260000 - 0xda27ffff]
[    0.310505] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.310510] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.310939] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.310969] pci 0000:02:00.0: expected upstream PCIe bridge, but 0000:00:1c.7 is type 0x4
[    0.310970] ------------[ cut here ]------------
[    0.310973] WARNING: at drivers/pci/search.c:49 pci_find_upstream_pcie_bridge+0x9e/0xb0()
[    0.310974] Hardware name:
[    0.310975] Modules linked in:
[    0.310977] Pid: 1, comm: swapper/0 Not tainted 3.6.0-rc3 #1
[    0.310977] Call Trace:
[    0.310981]  [<ffffffff8105d3cb>] ? warn_slowpath_common+0x7b/0xc0
[    0.310982]  [<ffffffff812b985e>] ? pci_find_upstream_pcie_bridge+0x9e/0xb0
[    0.310985]  [<ffffffff8148ad76>] ? intel_iommu_add_device+0x46/0x1d0
[    0.310987]  [<ffffffff8148767c>] ? add_iommu_group+0x2c/0x60
[    0.310989]  [<ffffffff81487650>] ? iommu_domain_free+0x20/0x20
[    0.310991]  [<ffffffff813a200d>] ? bus_for_each_dev+0x4d/0x80
[    0.310993]  [<ffffffff81487b24>] ? bus_set_iommu+0x44/0x60
[    0.310995]  [<ffffffff818a7ab8>] ? intel_iommu_init+0x9fd/0xa84
[    0.310998]  [<ffffffff81884124>] ? prepare_namespace+0x162/0x162
[    0.311000]  [<ffffffff81082a40>] ? lg_local_unlock+0x20/0x40
[    0.311002]  [<ffffffff811217ab>] ? sys_mkdirat+0x4b/0x70
[    0.311004]  [<ffffffff818867ae>] ? memblock_find_dma_reserve+0x13e/0x13e
[    0.311006]  [<ffffffff818867bc>] ? pci_iommu_init+0xe/0x37
[    0.311007]  [<ffffffff810001b9>] ? do_one_initcall+0x39/0x170
[    0.311009]  [<ffffffff81883b32>] ? kernel_init+0x132/0x1b6
[    0.311011]  [<ffffffff81883460>] ? loglevel+0x34/0x34
[    0.311012]  [<ffffffff815e0534>] ? kernel_thread_helper+0x4/0x10
[    0.311014]  [<ffffffff81883a00>] ? start_kernel+0x2e5/0x2e5
[    0.311015]  [<ffffffff815e0530>] ? gs_change+0xb/0xb
[    0.311018] ---[ end trace 1cbaf249bb3eb5b0 ]---
[    0.311022] pci 0000:03:00.0: expected upstream PCIe bridge, but 0000:00:1c.7 is type 0x4
Comment 7 Bjorn Helgaas 2012-08-23 15:58:39 UTC
Thanks for the report, Alessandro.  What kind of bridge is your 02:00.0 device?  So far we have:

03:00.0 ASMedia PCI bridge (John)
05:00.0 Tundra PCI bridge (Reuben)
02:00.0 ? PCI bridge (Alessandro)

The upstream device is 

00:1c.4 Intel 82801 PCI Bridge (John)
00:1c.3 Intel 6 Series/C200 PCI Express Root Port 4 (Reuben)
00:1c.7 Intel 7 Series/C210 PCI Express Root Port 8 (Alessandro)

I doubt the upstream device is at fault.  More likely the downstream device is really a PCIe device (a PCIe-to-PCI bridge) but just fails to report a PCIe capability.  I think this situation is likely too common to deal with via quirks, so we'll have to figure out a way to just make this work.
Comment 8 Alessandro Vietta 2012-08-23 18:32:26 UTC
Details (-vvv) of the device 00:02.0: Intel HD4000

00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09) (prog-if 00 [VGA controller])
        Subsystem: Intel Corporation Device 2031
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 44
        Region 0: Memory at f7800000 (64-bit, non-prefetchable) [size=4M]
        Region 2: Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Region 4: I/O ports at f000 [size=64]
        Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: feeff00c  Data: 41a1
        Capabilities: [d0] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [a4] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: i915
Comment 9 Bjorn Helgaas 2012-08-23 19:06:03 UTC
Wrong device, I think...  02:00.0 (not 00:02.0) is the interesting one.
Comment 10 Alessandro Vietta 2012-08-23 19:37:29 UTC
Yes, sorry:

02:00.0 PCI bridge: Integrated Technology Express, Inc. Device 8892 (rev 30) (prog-if 01 [Subtractive decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
        I/O behind bridge: 0000e000-0000efff
        Memory behind bridge: f7c00000-f7cfffff
        Capabilities: [90] Power Management version 2
        Capabilities: [a0] Subsystem: Intel Corporation Device 2031
Comment 11 Alex Williamson 2012-08-23 20:46:28 UTC
Created attachment 78351 [details]
Disable VT-d in presence of these devices

Would appreciate testing of this patch by those with the offending devices.  It should not be possible to enable VT-d on these systems with this patch as we don't know how devices behind these broken bridges will be tagged for the IOMMU.  This patch should override any kernel defaults to enable VT-d as well as command line options.  If we figure out more about these devices and how they tag transactions, we may be able to add quirks to individually re-enable them later.
Comment 12 Alessandro Vietta 2012-08-23 21:32:54 UTC
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ITE, 0x8892, quirk_bad_pcie_bridge);

The new function is triggered on boot as expected, disabling VT-d:

[    0.266889] Your hardware is broken, device 0000:02:00.0 claims to be a conventional PCI bridge device connected to an upstream PCIe device that is not a PCIe-to-PCI bridge.  Disabling VT-d support on this system.

These are the IOMMU related options that I have enabled into this kernel .config:
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y
CONFIG_INTEL_IOMMU=y
CONFIG_INTEL_IOMMU_DEFAULT_ON=y
CONFIG_INTEL_IOMMU_FLOPPY_WA=y

Ok, so no VT-d (if I'm not wrong it's enabled by default into the BIOS/firmware and I was just trying to play with it).
Comment 13 Reuben Farrelly 2012-08-23 23:51:07 UTC
Works for me too:

pci 0000:04:00.0: [1033:0194] type 00 class 0x0c0330
pci 0000:04:00.0: reg 10: [mem 0xf0100000-0xf0101fff 64bit]
pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.1: PCI bridge to [bus 04-04]
pci 0000:00:1c.1:   bridge window [mem 0xf0100000-0xf01fffff]
pci 0000:05:00.0: [10e3:8113] type 01 class 0x060401
Your hardware is broken, device 0000:05:00.0 claims to be a conventional PCI bridge device connected to an upstream PCIe device that is not a PCIe-to-PCI bridge.  Disabling VT-d support on this system.
pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:00:1c.3: PCI bridge to [bus 05-06]
pci 0000:05:00.0: PCI bridge to [bus 06-06] (subtractive decode)
pci 0000:05:00.0:   bridge window [??? 0x00000000 flags 0x0] (subtractive decode)
pci 0000:05:00.0:   bridge window [??? 0x00000000 flags 0x0] (subtractive decode)

Is this something we should/could be pushing back to Intel to fix in a future firmware update, or is it something hard set on the boards themselves?   Given there's a couple of us with Intel boards in this thread I wonder if it's something Intel should know about..
Comment 14 Javier Marcet 2012-09-02 17:04:22 UTC
I'm affected by this same bug with a Gigabyte H77M-D3H board but
curiously enough the xen hypervisor doesn't seem to be affected
by this issue. That is, the same kernel running as domain0 under
xen reports no issues with the IOMMU.
Comment 15 Javier Marcet 2012-09-02 21:53:52 UTC
By the way, the affected bridge on my board, a Gigabyte H77M-D3H, is this:

03:00.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev 30) (prog-if 01 [Subtractive decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=03, secondary=04, subordinate=04, sec-latency=32
        I/O behind bridge: 0000d000-0000dfff
        Memory behind bridge: f7a00000-f7afffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr+ DiscTmrStat- DiscTmrSERREn-
        Capabilities: [90] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=55mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [a0] Subsystem: Giga-byte Technology Device [1458:8892]
Comment 16 Reuben Farrelly 2013-01-05 22:57:26 UTC
What do we need to do to be able to close this report off - it has been open for a while with no activity and the (seemingly good) patch hasn't made it's way into any recent kernels AFAICT.

Is it likely to help if I contact Intel and open a case with them?  Or do we need more testers or confirmation of the problem being "fixed" with the patch?
Comment 17 Alex Williamson 2013-02-04 22:49:03 UTC
Created attachment 92521 [details]
Handle unadvertised PCIe bridges

Could folks experiencing this problem please test this patch and report results.  You should get a nice big warning in your dmesg that the hardware is broken and violates the PCIe spec.  Assuming vendors test Linux at all, it might help prevent such devices from existing.  Otherwise things should behave normally and VT-d should work.  If possible it would be great if someone with this hardware could plug in a PCI card to make it show up below one of these devices and see if it works with VT-d enabled.  Thanks.
Comment 18 Reuben Farrelly 2013-02-05 08:03:06 UTC
With this patch applied (but otherwise no changes to the system and no PCI cards in) I get this logged:

pci 0000:00:01.1: PCI bridge to [bus 02]
pci 0000:00:01.1:   bridge window [mem 0xf0200000-0xf04fffff]
pci 0000:00:1c.0: PCI bridge to [bus 03]
pci 0000:04:00.0: [1033:0194] type 00 class 0x0c0330
pci 0000:04:00.0: reg 10: [mem 0xf0100000-0xf0101fff 64bit]
pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.1: PCI bridge to [bus 04]
pci 0000:00:1c.1:   bridge window [mem 0xf0100000-0xf01fffff]
WARNING: Your hardware is broken, device (null) appears to be a
 Legacy PCI device attached directly to a PCIe device which is not a
 PCIe-to-PCI bridge.  Per section 7.8 of the PCI Express 3.0 spec, the
 PCI express capability structure is required for PCI express device
functions.
pci 0000:05:00.0: [10e3:8113] type 01 class 0x060401
pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:05:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
pci 0000:00:1c.3: PCI bridge to [bus 05-06]
Comment 19 Javier Marcet 2013-02-05 09:08:48 UTC
(In reply to comment #17)
> Created an attachment (id=92521) [details]
> Handle unadvertised PCIe bridges
> 
> Could folks experiencing this problem please test this patch and report
> results.  You should get a nice big warning in your dmesg that the hardware
> is
> broken and violates the PCIe spec.  Assuming vendors test Linux at all, it
> might help prevent such devices from existing.  Otherwise things should
> behave
> normally and VT-d should work.  If possible it would be great if someone with
> this hardware could plug in a PCI card to make it show up below one of these
> devices and see if it works with VT-d enabled.  Thanks.

I don't have the Gigabyte motherboard with the problem anymore, I exchanged it with an Asrock and I could not be happier. The Gigabyte had a lot of issue on Linux.
Comment 20 John Wehin 2013-02-08 13:59:06 UTC
Created attachment 92691 [details]
With this patch applied and no PCI logged

With this patch applied and no PCI logged
Comment 21 John Wehin 2013-02-08 14:00:36 UTC
Created attachment 92701 [details]
With this patch applied and PCI card logged
Comment 22 Kalin KOZHUHAROV 2013-04-10 16:10:44 UTC
Hello,

Seems like I hit this bug as well since 3.7.1, still here with 3.8.0.
My dmesg is attached for another bug #52061 as id=97991, here is the cut:

[    0.187785] ------------[ cut here ]------------
[    0.187883] WARNING: at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x56/0x70()
[    0.188030] Hardware name: dynabook R731/W2UB
[    0.188123] Modules linked in:
[    0.188256] Pid: 1, comm: swapper/0 Not tainted 3.8.0-r1 #1
[    0.188351] Call Trace:
[    0.188446]  [<ffffffff8105528f>] warn_slowpath_common+0x7e/0x96
[    0.188543]  [<ffffffff810552bc>] warn_slowpath_null+0x15/0x17
[    0.188640]  [<ffffffff8126dee2>] pci_find_upstream_pcie_bridge+0x56/0x70
[    0.188752]  [<ffffffff81376895>] intel_iommu_add_device+0x4b/0x169
[    0.188850]  [<ffffffff81373752>] add_iommu_group+0x3a/0x41
[    0.188946]  [<ffffffff81373718>] ? iommu_domain_alloc+0x68/0x68
[    0.189045]  [<ffffffff812ff011>] bus_for_each_dev+0x54/0x80
[    0.189142]  [<ffffffff81373e35>] bus_set_iommu+0x3d/0x44
[    0.189240]  [<ffffffff81ae0456>] intel_iommu_init+0xac3/0xb3c
[    0.189338]  [<ffffffff81127755>] ? sys_mkdirat+0x9b/0xc6
[    0.189436]  [<ffffffff81aae96e>] ? rd_load_disk+0xad/0xad
[    0.189533]  [<ffffffff81ab4c44>] ? memblock_find_dma_reserve+0x13f/0x13f
[    0.189631]  [<ffffffff81ab4c57>] pci_iommu_init+0x13/0x3e
[    0.189738]  [<ffffffff8100209a>] do_one_initcall+0x7a/0x134
[    0.189836]  [<ffffffff81aadbf2>] kernel_init_freeable+0x133/0x1bd
[    0.189933]  [<ffffffff81aad55d>] ? do_early_param+0x86/0x86
[    0.190031]  [<ffffffff814382bb>] ? rest_init+0x6f/0x6f
[    0.190127]  [<ffffffff814382c4>] kernel_init+0x9/0xd6
[    0.190225]  [<ffffffff8144db7c>] ret_from_fork+0x7c/0xb0
[    0.190321]  [<ffffffff814382bb>] ? rest_init+0x6f/0x6f
[    0.190420] ---[ end trace 935c32344ecc0b0a ]---

# lspci 
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:16.3 Serial controller: Intel Corporation 6 Series/C200 Series Chipset Family KT Controller (rev 04)
00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connection (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b4)
00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b4)
00:1c.2 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 3 (rev b4)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b4)
00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b4)
00:1c.6 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev b4)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation QM67 Express Chipset Family LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port SATA AHCI Controller (rev 04)
01:00.0 System peripheral: Ricoh Co Ltd PCIe SDXC/MMC Host Controller (rev 07)
02:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
02:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
04:00.0 Network controller: Intel Corporation Centrino Advanced-N + WiMAX 6250 [Kilmer Peak] (rev 5e)
05:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)
06:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)


Device 02:00.{0,1} is a eSATA CARD (AKE BC418), the machine (dynabook R731) is on a docking station with various connectors (LAN, USB, HDMI, etc.).
I couldn't quite figure out the supplied patch, I guess quite a few things changed in 3.8.0...
Comment 23 Alex Williamson 2013-05-10 21:27:40 UTC
Could folks experiencing this problem please test this series and report whether it solves the problem:

http://www.spinics.net/lists/linux-pci/msg22136.html
Comment 24 Reuben Farrelly 2013-05-10 23:47:32 UTC
The patch series fixes it for me and I had one of the problematic Intel boards.  Thanks Alex!
Comment 25 Alessandro Vietta 2013-05-12 12:23:48 UTC
Patches applied over kernel 3.9.1, no more visible errors.
Motherboard Intel DH77KC, BIOS 0108 (3/5/2013).

Thanks!
Comment 26 Justin 2013-05-13 10:23:57 UTC
Patches with 3.9.2 give no visible error here
Comment 27 Andreas Sturmlechner 2013-10-04 09:17:11 UTC
Patch applied over 3.11.3 and works!

Affected board: ASUS P9D WS (latest available BIOS 1406 08/30/2013), C226 Chipset
Comment 28 Kalin KOZHUHAROV 2013-10-15 07:46:51 UTC
I tried on a slightly different hardware with latest vanilla 3.11.5 and patched with the ones in comment 23. 

vanilla:

[    0.167443] ------------[ cut here ]------------
[    0.167539] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x54/0x6e()
[    0.167679] Modules linked in:
[    0.167812] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.11.5-r1 #1
[    0.167906] Hardware name: TOSHIBA dynabook R732/W3UG/dynabook R732/W3UG, BIOS Version 6.70   04/04/2013
[    0.169275]  0000000000000009 ffff88041bcc5cb8 ffffffff8143c0ad 00000000000000dc
[    0.169589]  0000000000000000 ffff88041bcc5cf8 ffffffff8104ab54 ffff88041bcc5cd8
[    0.169896]  ffffffff81266712 0000000000000000 ffff88041bf22000 ffff88041bf22098
[    0.170202] Call Trace:
[    0.170299]  [<ffffffff8143c0ad>] dump_stack+0x46/0x58
[    0.170392]  [<ffffffff8104ab54>] warn_slowpath_common+0x7c/0x96
[    0.170487]  [<ffffffff81266712>] ? pci_find_upstream_pcie_bridge+0x54/0x6e
[    0.170585]  [<ffffffff8136b69a>] ? bus_set_iommu+0x4a/0x4a
[    0.170678]  [<ffffffff8104ab83>] warn_slowpath_null+0x15/0x17
[    0.170774]  [<ffffffff81266712>] pci_find_upstream_pcie_bridge+0x54/0x6e
[    0.170869]  [<ffffffff8136e189>] intel_iommu_add_device+0x4c/0x1c7
[    0.170963]  [<ffffffff8136b69a>] ? bus_set_iommu+0x4a/0x4a
[    0.171056]  [<ffffffff8136b6cf>] add_iommu_group+0x35/0x43
[    0.171150]  [<ffffffff812ef370>] bus_for_each_dev+0x57/0x89
[    0.171244]  [<ffffffff8136b693>] bus_set_iommu+0x43/0x4a
[    0.171348]  [<ffffffff81aea61f>] intel_iommu_init+0xa45/0xb54
[    0.171444]  [<ffffffff81ab28dd>] pci_iommu_init+0x13/0x3e
[    0.171537]  [<ffffffff81ab28ca>] ? memblock_find_dma_reserve+0x133/0x133
[    0.171633]  [<ffffffff81000267>] do_one_initcall+0x7f/0x104
[    0.171728]  [<ffffffff8106418a>] ? parse_args+0x190/0x24b
[    0.171822]  [<ffffffff81aaddb5>] kernel_init_freeable+0x138/0x1c7
[    0.171917]  [<ffffffff81aad71a>] ? do_early_param+0x8c/0x8c
[    0.172011]  [<ffffffff81435570>] ? rest_init+0x74/0x74
[    0.172104]  [<ffffffff81435579>] kernel_init+0x9/0xd6
[    0.172196]  [<ffffffff8144132c>] ret_from_fork+0x7c/0xb0
[    0.172294]  [<ffffffff81435570>] ? rest_init+0x74/0x74
[    0.172391] ---[ end trace 23884855cb05242e ]---

patched:
[    0.169380] ------------[ cut here ]------------
[    0.169476] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x5a/0x72()
[    0.169617] Modules linked in:
[    0.169750] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.11.5-r2 #2
[    0.169844] Hardware name: TOSHIBA dynabook R732/W3UG/dynabook R732/W3UG, BIOS Version 6.70   04/04/2013
[    0.171213]  0000000000000009 ffff88041bcc5cb8 ffffffff8143c18d 00000000000000dc
[    0.171528]  0000000000000000 ffff88041bcc5cf8 ffffffff8104ab54 ffff88041bcc5cd8
[    0.171834]  ffffffff812667c4 0000000000000000 ffff88041be1a000 ffff88041be1a098
[    0.172140] Call Trace:
[    0.172232]  [<ffffffff8143c18d>] dump_stack+0x46/0x58
[    0.172332]  [<ffffffff8104ab54>] warn_slowpath_common+0x7c/0x96
[    0.172430]  [<ffffffff812667c4>] ? pci_find_upstream_pcie_bridge+0x5a/0x72
[    0.172526]  [<ffffffff8136b76e>] ? bus_set_iommu+0x4a/0x4a
[    0.172620]  [<ffffffff8104ab83>] warn_slowpath_null+0x15/0x17
[    0.172714]  [<ffffffff812667c4>] pci_find_upstream_pcie_bridge+0x5a/0x72
[    0.172809]  [<ffffffff8136e260>] intel_iommu_add_device+0x4c/0x1ca
[    0.172903]  [<ffffffff8136b76e>] ? bus_set_iommu+0x4a/0x4a
[    0.172997]  [<ffffffff8136b7a3>] add_iommu_group+0x35/0x43
[    0.173091]  [<ffffffff812ef444>] bus_for_each_dev+0x57/0x89
[    0.173185]  [<ffffffff8136b767>] bus_set_iommu+0x43/0x4a
[    0.173290]  [<ffffffff81aea61f>] intel_iommu_init+0xa45/0xb54
[    0.173385]  [<ffffffff81ab28dd>] pci_iommu_init+0x13/0x3e
[    0.173479]  [<ffffffff81ab28ca>] ? memblock_find_dma_reserve+0x133/0x133
[    0.173576]  [<ffffffff81000267>] do_one_initcall+0x7f/0x104
[    0.173670]  [<ffffffff8106418a>] ? parse_args+0x190/0x24b
[    0.173765]  [<ffffffff81aaddb5>] kernel_init_freeable+0x138/0x1c7
[    0.173859]  [<ffffffff81aad71a>] ? do_early_param+0x8c/0x8c
[    0.173955]  [<ffffffff81435650>] ? rest_init+0x74/0x74
[    0.174047]  [<ffffffff81435659>] kernel_init+0x9/0xd6
[    0.174140]  [<ffffffff8144142c>] ret_from_fork+0x7c/0xb0
[    0.174233]  [<ffffffff81435650>] ? rest_init+0x74/0x74
[    0.174334] ---[ end trace afb127d33a1e0828 ]---

# lspci 
00:00.0 Host bridge: Intel Corporation 3rd Gen Core processor DRAM Controller (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04)
00:16.3 Serial controller: Intel Corporation 7 Series/C210 Series Chipset Family KT Controller (rev 04)
00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connection (rev 04)
00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)
00:1c.1 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 (rev c4)
00:1c.2 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 3 (rev c4)
00:1c.5 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 6 (rev c4)
00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation QM77 Express Chipset LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 7 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
01:00.0 System peripheral: Ricoh Co Ltd PCIe SDXC/MMC Host Controller (rev 07)
02:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
02:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
04:00.0 Network controller: Intel Corporation Centrino Advanced-N + WiMAX 6250 [Kilmer Peak] (rev 5e)
05:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)


Booting (patched), then inserting the eSATA card, I get similar output:
[   45.201580] ACPI Error: Method parse/execution failed [\_GPE._L01] (Node ffff88041bc607f8), AE_AML_INFINITE_LOOP (20130517/psparse-536)
[   45.201587] ACPI Exception: AE_AML_INFINITE_LOOP, while evaluating GPE method [_L01] (20130517/evgpe-579)
[   45.479769] ACPI Error: Method parse/execution failed [\_GPE._L01] (Node ffff88041bc607f8), AE_AML_INFINITE_LOOP (20130517/psparse-536)
[   45.479777] ACPI Exception: AE_AML_INFINITE_LOOP, while evaluating GPE method [_L01] (20130517/evgpe-579)
[   45.781017] ACPI Error: Method parse/execution failed [\_GPE._L01] (Node ffff88041bc607f8), AE_AML_INFINITE_LOOP (20130517/psparse-536)
[   45.781025] ACPI Exception: AE_AML_INFINITE_LOOP, while evaluating GPE method [_L01] (20130517/evgpe-579)
[   46.062741] ACPI Error: Method parse/execution failed [\_GPE._L01] (Node ffff88041bc607f8), AE_AML_INFINITE_LOOP (20130517/psparse-536)
[   46.062748] ACPI Exception: AE_AML_INFINITE_LOOP, while evaluating GPE method [_L01] (20130517/evgpe-579)
[   46.360495] ACPI Error: Method parse/execution failed [\_GPE._L01] (Node ffff88041bc607f8), AE_AML_INFINITE_LOOP (20130517/psparse-536)
[   46.360502] ACPI Exception: AE_AML_INFINITE_LOOP, while evaluating GPE method [_L01] (20130517/evgpe-579)
[   46.605812] ACPI: Device does not support D3cold
[   46.605897] pci 0000:02:00.0: [197b:2363] type 00 class 0x010601
[   46.606034] pci 0000:02:00.0: reg 0x24: [mem 0x00000000-0x00001fff]
[   46.606054] pci 0000:02:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[   46.606130] pci 0000:02:00.0: PME# supported from D3hot
[   46.606164] pci 0000:02:00.0: System wakeup disabled by ACPI
[   46.606233] pci 0000:02:00.1: [197b:2363] type 00 class 0x010185
[   46.606267] pci 0000:02:00.1: reg 0x10: [io  0x0000-0x0007]
[   46.606286] pci 0000:02:00.1: reg 0x14: [io  0x0000-0x0003]
[   46.606306] pci 0000:02:00.1: reg 0x18: [io  0x0000-0x0007]
[   46.606326] pci 0000:02:00.1: reg 0x1c: [io  0x0000-0x0003]
[   46.606345] pci 0000:02:00.1: reg 0x20: [io  0x0000-0x000f]
[   46.606473] ------------[ cut here ]------------
[   46.606479] WARNING: CPU: 0 PID: 1270 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x5a/0x72()
[   46.606481] Modules linked in: ipv6 snd_usb_audio snd_usbmidi_lib hid_generic snd_rawmidi usbhid snd_seq_device dm_mod uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_core videodev i2400m_usb i2400m wimax fbcon bitblit softcursor font joydev snd_hda_codec_hdmi snd_hda_codec_realtek arc4 iwldvm mac80211 i915 x86_pkg_temp_thermal crc32c_intel iwlwifi i2c_algo_bit snd_hda_intel intel_agp snd_hda_codec intel_gtt sr_mod cfg80211 sdhci_pci cdrom sdhci mmc_core toshiba_acpi sparse_keymap led_class rfkill snd_hwdep drm_kms_helper ehci_pci snd_pcm e1000e ehci_hcd xhci_hcd drm snd_page_alloc snd_timer agpgart usbcore wmi ptp snd video pps_core usb_common
[   46.606521] CPU: 0 PID: 1270 Comm: kworker/0:2 Not tainted 3.11.5-r2 #2
[   46.606522] Hardware name: TOSHIBA dynabook R732/W3UG/dynabook R732/W3UG, BIOS Version 6.70   04/04/2013
[   46.606526] Workqueue: kacpi_hotplug _handle_hotplug_event_bridge
[   46.606527]  0000000000000009 ffff880418ff59a8 ffffffff8143c18d 00000000000000f6
[   46.606530]  0000000000000000 ffff880418ff59e8 ffffffff8104ab54 0000000000000000
[   46.606532]  ffffffff812667c4 0000000000000000 ffff880418ebc000 ffff880418ebc098
[   46.606534] Call Trace:
[   46.606539]  [<ffffffff8143c18d>] dump_stack+0x46/0x58
[   46.606542]  [<ffffffff8104ab54>] warn_slowpath_common+0x7c/0x96
[   46.606545]  [<ffffffff812667c4>] ? pci_find_upstream_pcie_bridge+0x5a/0x72
[   46.606548]  [<ffffffff8104ab83>] warn_slowpath_null+0x15/0x17
[   46.606550]  [<ffffffff812667c4>] pci_find_upstream_pcie_bridge+0x5a/0x72
[   46.606555]  [<ffffffff8136e260>] intel_iommu_add_device+0x4c/0x1ca
[   46.606558]  [<ffffffff8116a46e>] ? sysfs_add_one+0x16/0xa5
[   46.606562]  [<ffffffff8136b836>] iommu_bus_notifier+0x30/0xa5
[   46.606565]  [<ffffffff81069b80>] notifier_call_chain+0x32/0x5c
[   46.606568]  [<ffffffff81069cb3>] __blocking_notifier_call_chain+0x41/0x5a
[   46.606571]  [<ffffffff81069cdb>] blocking_notifier_call_chain+0xf/0x11
[   46.606575]  [<ffffffff812ee7b2>] device_add+0x436/0x624
[   46.606578]  [<ffffffff81262c7a>] ? pci_find_ext_capability+0xd/0xf
[   46.606580]  [<ffffffff81260bd7>] pci_device_add+0x127/0x14c
[   46.606583]  [<ffffffff814367cb>] pci_scan_single_device+0x93/0xa1
[   46.606585]  [<ffffffff81260c7c>] pci_scan_slot+0x80/0xf9
[   46.606586]  [<ffffffff81437665>] enable_device+0xeb/0x394
[   46.606588]  [<ffffffff81274336>] ? get_slot_status+0x7a/0xa9
[   46.606592]  [<ffffffff81275218>] acpiphp_enable_slot+0xc3/0x128
[   46.606594]  [<ffffffff812753e9>] acpiphp_check_bridge+0x6b/0xd1
[   46.606597]  [<ffffffff812757d6>] _handle_hotplug_event_bridge+0x10a/0x1fc
[   46.606606]  [<ffffffff8105dc7f>] ? pwq_activate_delayed_work+0x47/0x6a
[   46.606608]  [<ffffffff8106038c>] process_one_work+0x1f4/0x36f
[   46.606610]  [<ffffffff8106094c>] worker_thread+0x1cd/0x2c4
[   46.606612]  [<ffffffff8106077f>] ? rescuer_thread+0x24e/0x24e
[   46.606615]  [<ffffffff810658bc>] kthread+0xb5/0xbd
[   46.606617]  [<ffffffff81065807>] ? kthread_freezable_should_stop+0x48/0x48
[   46.606621]  [<ffffffff8144142c>] ret_from_fork+0x7c/0xb0
[   46.606623]  [<ffffffff81065807>] ? kthread_freezable_should_stop+0x48/0x48
[   46.606626] ---[ end trace 76ba6fcb9ecb9e9c ]---
[   46.606673] pci 0000:02:00.0: BAR 6: assigned [mem 0xe0400000-0xe040ffff pref]
[   46.606675] pci 0000:02:00.0: BAR 5: assigned [mem 0xe2600000-0xe2601fff]
[   46.606682] pci 0000:02:00.1: BAR 4: assigned [io  0x2000-0x200f]
[   46.606688] pci 0000:02:00.1: BAR 0: assigned [io  0x2010-0x2017]
[   46.606694] pci 0000:02:00.1: BAR 2: assigned [io  0x2018-0x201f]
[   46.606703] pci 0000:02:00.1: BAR 1: assigned [io  0x2020-0x2023]
[   46.606710] pci 0000:02:00.1: BAR 3: assigned [io  0x2024-0x2027]
[   46.606722] pci 0000:02:00.0: no hotplug settings from platform
[   46.606728] pci 0000:02:00.1: no hotplug settings from platform
[   46.606732] pci 0000:02:00.1: using default PCI settings
[   46.606787] ahci 0000:02:00.0: enabling device (0000 -> 0002)
[   46.606928] ahci 0000:02:00.0: AHCI 0001.0000 32 slots 2 ports 3 Gbps 0x3 impl SATA mode
[   46.606931] ahci 0000:02:00.0: flags: 64bit ncq pm led clo pmp pio slum part 
[   46.607182] scsi6 : ahci
[   46.607248] scsi7 : ahci
[   46.607292] ata7: SATA max UDMA/133 abar m8192@0xe2600000 port 0xe2600100 irq 17
[   46.607296] ata8: SATA max UDMA/133 abar m8192@0xe2600000 port 0xe2600180 irq 17
[   46.608876] pata_jmicron 0000:02:00.1: enabling device (0000 -> 0001)
[   46.608946] pata_jmicron 0000:02:00.1: setting latency timer to 64
[   46.609278] scsi8 : pata_jmicron
[   46.609446] scsi9 : pata_jmicron
[   46.609469] ata9: PATA max UDMA/100 cmd 0x2010 ctl 0x2020 bmdma 0x2000 irq 18
[   46.609470] ata10: PATA max UDMA/100 cmd 0x2018 ctl 0x2024 bmdma 0x2008 irq 18
[   46.912371] ata7: SATA link down (SStatus 0 SControl 300)
[   46.912424] ata8: SATA link down (SStatus 0 SControl 300)

# lspci -s 02:00 -vv
02:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03) (prog-if 01 [AHCI 1.0])
	Subsystem: JMicron Technology Corp. JMB363 SATA/IDE Controller
	Physical Slot: 1
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 17
	Region 5: Memory at e2600000 (32-bit, non-prefetchable) [size=8K]
	[virtual] Expansion ROM at e0400000 [disabled] [size=64K]
	Capabilities: [68] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] Express (v1) Legacy Endpoint, MSI 01
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr+ UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <1us, L1 <16us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Kernel driver in use: ahci

02:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03) (prog-if 85 [Master SecO PriO])
	Subsystem: JMicron Technology Corp. JMB363 SATA/IDE Controller
	Physical Slot: 1
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin B routed to IRQ 18
	Region 0: I/O ports at 2010 [size=8]
	Region 1: I/O ports at 2020 [size=4]
	Region 2: I/O ports at 2018 [size=8]
	Region 3: I/O ports at 2024 [size=4]
	Region 4: I/O ports at 2000 [size=16]
	Capabilities: [68] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pata_jmicron
	Kernel modules: pata_jmicron
Comment 29 Andrey Nazarov 2013-11-16 16:55:58 UTC
(In reply to Alex Williamson from comment #23)
> Could folks experiencing this problem please test this series and report
> whether it solves the problem:
> 
> http://www.spinics.net/lists/linux-pci/msg22136.html

This series prevent the boot time warning from showing up, however attempting to actually passthrough a FireWire controller behind the offending PCIe-to-PCI bridge using VFIO results in the following oops:

[  319.773836] VFIO - User Level meta-driver version: 0.3
[  379.948753] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
[  379.948782] IP: [<ffffffff812c3b65>] domain_context_mapping+0x85/0x106
[  379.948806] PGD 0 
[  379.948813] Oops: 0000 [#1] SMP 
[  379.948824] Modules linked in: vfio_pci vfio_iommu_type1 vfio ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables parport_pc ppdev lp parport nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc fuse snd_hda_codec_hdmi snd_hda_codec_realtek x86_pkg_temp_thermal coretemp snd_hda_intel kvm_intel snd_hda_codec kvm snd_hwdep snd_pcm crc32c_intel ghash_clmulni_intel aesni_intel snd_page_alloc aes_x86_64 ablk_helper snd_seq snd_seq_device snd_timer snd cryptd i915 drm_kms_helper lrw drm i2c_algo_bit iTCO_wdt iTCO_vendor_support soundcore i2c_i801 psmouse i2c_core gf128mul lpc_ich mfd_core video button mei_me mei processor evdev glue_helper serio_raw microcode ext4 crc16 jbd2 mbcache dm_mod sg sd_mod crc_t10dif crct10dif_common hid_generic usbhid hid ahci libahci libata scsi_mod ehci_pci ehci_hcd xhci_hcd e1000e usbcore ptp pps_core usb_common thermal fan thermal_sys
[  379.949111] CPU: 1 PID: 2700 Comm: qemu-system-x86 Not tainted 3.12.0-rc7-00003-g8eab36e #4
[  379.949133] Hardware name: Supermicro X10SAE/X10SAE, BIOS 1.1 08/13/2013
[  379.949152] task: ffff8807f38337f0 ti: ffff8807f3b18000 task.ti: ffff8807f3b18000
[  379.949171] RIP: 0010:[<ffffffff812c3b65>]  [<ffffffff812c3b65>] domain_context_mapping+0x85/0x106
[  379.949196] RSP: 0018:ffff8807f3b19dd8  EFLAGS: 00010207
[  379.949210] RAX: 0000000000000000 RBX: ffff8807f7087000 RCX: 0000000000000033
[  379.949229] RDX: 0000000000330033 RSI: 0000000000000286 RDI: 0000000000000286
[  379.949247] RBP: ffff8807f7087000 R08: 0000000000000000 R09: 00000000006e9202
[  379.949265] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8807f4595340
[  379.949284] R13: 0000000000000000 R14: ffffffff812bab2c R15: ffff8807f68fb4c8
[  379.949302] FS:  00007fdfe626b900(0000) GS:ffff88081fa40000(0000) knlGS:0000000000000000
[  379.949323] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  379.949338] CR2: 0000000000000010 CR3: 00000007f42c9000 CR4: 00000000001427e0
[  379.949357] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  379.949375] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  379.949393] Stack:
[  379.949399]  0000000000000000 ffff8807f4597540 ffff8807f7087000 ffff8807f4595340
[  379.949422]  0000000000000000 ffffffff812c3c9d ffff8807f6639300 ffff8807f45b1c08
[  379.949445]  ffff8807f45b1c18 ffff8807f4369f00 ffffffff812babf0 00000000000080d0
[  379.949467] Call Trace:
[  379.949476]  [<ffffffff812c3c9d>] ? domain_add_dev_info+0xb7/0xfd
[  379.949495]  [<ffffffff812babf0>] ? iommu_group_for_each_dev+0x36/0x5b
[  379.949514]  [<ffffffffa0545511>] ? vfio_iommu_type1_attach_group+0x85/0xd8 [vfio_iommu_type1]
[  379.949537]  [<ffffffffa053b403>] ? vfio_fops_unl_ioctl+0x1bb/0x2bd [vfio]
[  379.949557]  [<ffffffff81123490>] ? vfs_ioctl+0x1b/0x28
[  379.949571]  [<ffffffff81123ba6>] ? do_vfs_ioctl+0x327/0x3e2
[  379.949588]  [<ffffffff81105925>] ? kmem_cache_free+0x3c/0x72
[  379.949604]  [<ffffffff81123caf>] ? SyS_ioctl+0x4e/0x7b
[  379.949619]  [<ffffffff8139a8b9>] ? system_call_fastpath+0x16/0x1b
[  379.949635] Code: 1f 81 48 89 ef e8 e7 1d f3 ff 48 85 c0 48 89 c3 75 04 48 8b 1c 24 48 8b 45 10 48 8b 40 38 48 89 04 24 48 8b 04 24 48 39 d8 74 31 <48> 8b 70 10 0f b6 48 38 45 89 e8 4c 89 e7 48 8b 86 c8 00 00 00 
[  379.949738] RIP  [<ffffffff812c3b65>] domain_context_mapping+0x85/0x106
[  379.949755]  RSP <ffff8807f3b19dd8>
[  379.949764] CR2: 0000000000000010
[  379.955446] ---[ end trace 8de99f9a31b5b2ec ]---

This happens when the bridge itself is passed into domain_context_mapping. In this case pci_walk_up_to_first_match returns pdev on the first match and inner loop in domain_context_mapping doesn't terminate, hitting root bus with NULL parent.

Topology of this board is as follows:

-[0000:00]-+-00.0
           +-01.0-[01]--+-00.0
           |            \-00.1
           +-02.0
           +-03.0
           +-14.0
           +-16.0
           +-19.0
           +-1b.0
           +-1c.0-[02]--
           +-1c.3-[03]----00.0
           +-1c.5-[04-05]----00.0-[05]----03.0
           +-1c.6-[06]----00.0
           +-1c.7-[07]----00.0
           +-1f.0
           +-1f.2
           +-1f.3
           \-1f.6

00:1c.5 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #6 (rev d4)
04:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8113 (rev 01)
05:03.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22A IEEE-1394a-2000 Controller (PHY/Link) [iOHCI-Lynx]
Comment 30 Alan 2013-12-03 15:34:17 UTC
*** Bug 61581 has been marked as a duplicate of this bug. ***
Comment 31 Ingo Theiss 2014-02-03 17:25:28 UTC
Using Kernel 3.14-rc1 and still getting the warning.

Intel i7 + Asrock Z87 PRO4

---
[    0.336268] DMAR: No ATSR found
[    0.336291] IOMMU 0 0xfed90000: using Queued invalidation
[    0.336292] IOMMU 1 0xfed91000: using Queued invalidation
[    0.336294] IOMMU: Setting RMRR:
[    0.336304] IOMMU: Setting identity map for device 0000:00:02.0 [0xc7800000 - 0xcf9fffff]
[    0.336946] IOMMU: Setting identity map for device 0000:00:1d.0 [0xc40fa000 - 0xc4106fff]
[    0.336971] IOMMU: Setting identity map for device 0000:00:1a.0 [0xc40fa000 - 0xc4106fff]
[    0.336989] IOMMU: Setting identity map for device 0000:00:14.0 [0xc40fa000 - 0xc4106fff]
[    0.337003] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.337010] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.337122] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.337217] ------------[ cut here ]------------
[    0.337222] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x4d/0x5e()
[    0.337224] Modules linked in:
[    0.337227] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.0-rc1-spoc #1
[    0.337229] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z87 Pro4, BIOS P1.70 07/04/2013
[    0.337231]  0000b42000000649 ffffffff813bf921 0000000000000000 ffffffff81045f65
[    0.337234]  ffffffff8120b3be 0000000000000000 ffff88041a792000 ffff88041a792098
[    0.337236]  0000000000000000 ffffffff8120b3be ffffffff81683290 ffffffff812ea47b
[    0.337239] Call Trace:
[    0.337243]  [<ffffffff813bf921>] ? dump_stack+0x41/0x51
[    0.337247]  [<ffffffff81045f65>] ? warn_slowpath_common+0x74/0x89
[    0.337249]  [<ffffffff8120b3be>] ? pci_find_upstream_pcie_bridge+0x4d/0x5e
[    0.337252]  [<ffffffff8120b3be>] ? pci_find_upstream_pcie_bridge+0x4d/0x5e
[    0.337254]  [<ffffffff812ea47b>] ? intel_iommu_add_device+0x43/0x1ab
[    0.337257]  [<ffffffff812e345c>] ? bus_set_iommu+0x43/0x43
[    0.337259]  [<ffffffff812e348c>] ? add_iommu_group+0x30/0x3c
[    0.337262]  [<ffffffff812a5830>] ? bus_for_each_dev+0x6e/0x78
[    0.337264]  [<ffffffff812e3457>] ? bus_set_iommu+0x3e/0x43
[    0.337267]  [<ffffffff81700e91>] ? intel_iommu_init+0xc80/0xd5a
[    0.337271]  [<ffffffff810d99a5>] ? free_reserved_area+0xfb/0x10d
[    0.337274]  [<ffffffff816c6f22>] ? pci_iommu_init+0xe/0x37
[    0.337276]  [<ffffffff816c6f14>] ? memblock_find_dma_reserve+0x131/0x131
[    0.337279]  [<ffffffff810020da>] ? do_one_initcall+0x88/0x123
[    0.337282]  [<ffffffff8105d95d>] ? parse_args+0x182/0x236
[    0.337285]  [<ffffffff816bef36>] ? kernel_init_freeable+0x172/0x1f4
[    0.337287]  [<ffffffff816be761>] ? do_early_param+0x81/0x81
[    0.337290]  [<ffffffff813b3caa>] ? rest_init+0x6e/0x6e
[    0.337292]  [<ffffffff813b3caf>] ? kernel_init+0x5/0xff
[    0.337295]  [<ffffffff813c8e8c>] ? ret_from_fork+0x7c/0xb0
[    0.337297]  [<ffffffff813b3caa>] ? rest_init+0x6e/0x6e
[    0.337301] ---[ end trace 90663f41996c57f8 ]---
---
Comment 32 Andreas Klauer 2014-02-15 18:54:42 UTC
More of the same, kernel 3.13.3, Haswell Asus-Z87-Plus with current latest bios

[    0.315293] ------------[ cut here ]------------
[    0.315386] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x6b/0x80()
[    0.315526] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.13.3 #1
[    0.315613] Hardware name: ASUS All Series/Z87-PLUS, BIOS 1707 12/13/2013
[    0.315703]  0000000000000009 ffffffff816b2a86 0000000000000000 ffffffff810d3278
[    0.315960]  ffff88041d4b6898 ffff88041d4b6800 ffff88041d4b6898 ffffffff8199e140
[    0.316217]  0000000000000001 ffffffff8133583b ffffffff8156ebc3 ffff88041d4b6898
[    0.316482] Call Trace:
[    0.316566]  [<ffffffff816b2a86>] ? dump_stack+0x41/0x51
[    0.316654]  [<ffffffff810d3278>] ? warn_slowpath_common+0x68/0x80
[    0.316742]  [<ffffffff8133583b>] ? pci_find_upstream_pcie_bridge+0x6b/0x80
[    0.316832]  [<ffffffff8156ebc3>] ? intel_iommu_add_device+0x43/0x210
[    0.316922]  [<ffffffff8156bb40>] ? iommu_bus_notifier+0xc0/0xc0
[    0.317010]  [<ffffffff8156bb60>] ? add_iommu_group+0x20/0x50
[    0.317098]  [<ffffffff8143afb3>] ? bus_for_each_dev+0x53/0x90
[    0.317186]  [<ffffffff8156c27e>] ? bus_set_iommu+0x3e/0x50
[    0.317275]  [<ffffffff819fbab2>] ? intel_iommu_init+0x9ce/0xac4
[    0.317367]  [<ffffffff819d64a9>] ? memblock_find_dma_reserve+0x135/0x135
[    0.317456]  [<ffffffff819d64b7>] ? pci_iommu_init+0xe/0x37
[    0.317543]  [<ffffffff819d0dd3>] ? do_one_initcall+0x78/0xfd
[    0.317631]  [<ffffffff810ebe70>] ? parse_args+0x120/0x320
[    0.317718]  [<ffffffff819d0f60>] ? kernel_init_freeable+0x108/0x18a
[    0.317805]  [<ffffffff819d07c5>] ? do_early_param+0x81/0x81
[    0.317894]  [<ffffffff816ac0e0>] ? rest_init+0x70/0x70
[    0.317981]  [<ffffffff816ac0e5>] ? kernel_init+0x5/0x110
[    0.318067]  [<ffffffff816b967c>] ? ret_from_fork+0x7c/0xb0
[    0.318155]  [<ffffffff816ac0e0>] ? rest_init+0x70/0x70
[    0.318243] ---[ end trace b910a9416ec47712 ]---
Comment 33 Daniel Gottschalk 2014-02-18 20:32:33 UTC
Kernel 3.13.3, Haswell Asus P9D WS, latest bios

[    2.458391] ------------[ cut here ]------------
[    2.467654] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x53/0x70()
[    2.486470] Modules linked in:
[    2.492678] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.13.3 #5
[    2.504541] Hardware name: ASUS All Series/P9D WS, BIOS 1505 10/04/2013
[    2.517798]  0000000000000009 ffff88040f121cd8 ffffffff813d1061 0000000000000eb8
[    2.532876]  0000000000000000 ffff88040f121d18 ffffffff81041812 ffff88040f121d38
[    2.547949]  ffff88040f364898 ffff88040f364800 ffff88040f364898 0000000000000292
[    2.563024] Call Trace:
[    2.567957]  [<ffffffff813d1061>] dump_stack+0x46/0x58
[    2.578269]  [<ffffffff81041812>] warn_slowpath_common+0x82/0xb0
[    2.590312]  [<ffffffff81041855>] warn_slowpath_null+0x15/0x20
[    2.602008]  [<ffffffff811bd133>] pci_find_upstream_pcie_bridge+0x53/0x70
[    2.615611]  [<ffffffff8131de3c>] intel_iommu_add_device+0x4c/0x230
[    2.628173]  [<ffffffff8131aeb0>] ? bus_set_iommu+0x60/0x60
[    2.639348]  [<ffffffff8131aede>] add_iommu_group+0x2e/0x60
[    2.650530]  [<ffffffff812dc145>] bus_for_each_dev+0x55/0x90
[    2.661872]  [<ffffffff8131ae9c>] bus_set_iommu+0x4c/0x60
[    2.672701]  [<ffffffff816fb44c>] intel_iommu_init+0xb9d/0xcc4
[    2.684399]  [<ffffffff8103616f>] ? free_init_pages+0x7f/0xb0
[    2.695919]  [<ffffffff816d1606>] ? initrd_load+0x3f/0x3f
[    2.706747]  [<ffffffff816d2032>] ? maybe_link.part.5+0x100/0x100
[    2.718967]  [<ffffffff816d59d5>] ? memblock_find_dma_reserve+0x13d/0x13d
[    2.732569]  [<ffffffff816d59e8>] pci_iommu_init+0x13/0x3e
[    2.743570]  [<ffffffff81000312>] do_one_initcall+0x102/0x160
[    2.755092]  [<ffffffff8107209e>] ? __wake_up+0x4e/0x70
[    2.765573]  [<ffffffff816d0f90>] kernel_init_freeable+0x108/0x19c
[    2.777963]  [<ffffffff816d0866>] ? do_early_param+0x8c/0x8c
[    2.789313]  [<ffffffff813cb350>] ? rest_init+0x80/0x80
[    2.799794]  [<ffffffff813cb359>] kernel_init+0x9/0x120
[    2.810278]  [<ffffffff813d6d0c>] ret_from_fork+0x7c/0xb0
[    2.821108]  [<ffffffff813cb350>] ? rest_init+0x80/0x80
[    2.831591] ---[ end trace 3c8c03b92fd45291 ]---
Comment 34 Marcus Schmidt 2014-02-21 15:32:58 UTC
Kernel 3.13.0, Asrock Z77 Fatal1ty Professional, BIOS P1.70B

------------[ cut here ]------------
[    0.386934] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x77/0x90()
[    0.386935] Modules linked in:
[    0.386938] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.13.0-1-mainline #1
[    0.386939] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77 Professional, BIOS P1.70B 11/12/2013
[    0.386940]  0000000000000009 ffff880803469d20 ffffffff814f5cc4 0000000000000000
[    0.386942]  ffff880803469d58 ffffffff81061abd ffff8808032b3098 ffff8808032b3000
[    0.386944]  ffff8808032b3098 0000000000000000 ffff88080e8097e0 ffff880803469d68
[    0.386946] Call Trace:
[    0.386951]  [<ffffffff814f5cc4>] dump_stack+0x4d/0x6f
[    0.386954]  [<ffffffff81061abd>] warn_slowpath_common+0x7d/0xa0
[    0.386956]  [<ffffffff81061b9a>] warn_slowpath_null+0x1a/0x20
[    0.386958]  [<ffffffff812c9c07>] pci_find_upstream_pcie_bridge+0x77/0x90
[    0.386961]  [<ffffffff813e429d>] intel_iommu_add_device+0x4d/0x220
[    0.386964]  [<ffffffff813da620>] ? bus_set_iommu+0x50/0x50
[    0.386965]  [<ffffffff813da64a>] add_iommu_group+0x2a/0x50
[    0.386968]  [<ffffffff8138c633>] bus_for_each_dev+0x63/0xa0
[    0.386970]  [<ffffffff813da618>] bus_set_iommu+0x48/0x50
[    0.386973]  [<ffffffff8192d0cb>] intel_iommu_init+0xaad/0xba2
[    0.386976]  [<ffffffff81052355>] ? free_init_pages+0x75/0xa0
[    0.386979]  [<ffffffff818e61c5>] ? maybe_link.part.1+0x10e/0x10e
[    0.386982]  [<ffffffff818eed6a>] ? memblock_find_dma_reserve+0x124/0x124
[    0.386983]  [<ffffffff818eed7c>] pci_iommu_init+0x12/0x3c
[    0.386986]  [<ffffffff8100214a>] do_one_initcall+0xfa/0x1b0
[    0.386989]  [<ffffffff810818b5>] ? parse_args+0x225/0x3f0
[    0.386991]  [<ffffffff818e50da>] kernel_init_freeable+0x17b/0x200
[    0.386993]  [<ffffffff818e48d0>] ? do_early_param+0x88/0x88
[    0.386996]  [<ffffffff814e8bf0>] ? rest_init+0x90/0x90
[    0.386998]  [<ffffffff814e8bfe>] kernel_init+0xe/0x130
[    0.387000]  [<ffffffff8150407c>] ret_from_fork+0x7c/0xb0
[    0.387002]  [<ffffffff814e8bf0>] ? rest_init+0x90/0x90
[    0.387006] ---[ end trace d5a6f7ddd256cb5c ]---

Topology :

-[0000:00]-+-00.0
           +-01.0-[01]--+-00.0
           |            \-00.1
           +-02.0
           +-14.0
           +-16.0
           +-1a.0
           +-1b.0
           +-1c.0-[02]----00.0
           +-1c.7-[03-0c]----00.0-[04-0c]--+-01.0-[05]----00.0
           |                               +-04.0-[06-07]----00.0-[07]--
           |                               +-05.0-[08]----00.0
           |                               +-06.0-[09]----00.0
           |                               +-07.0-[0a]----00.0
           |                               +-08.0-[0b]----00.0
           |                               \-09.0-[0c]--+-00.0
           |                                            \-00.1
           +-1d.0
           +-1f.0
           +-1f.2
           \-1f.3
Comment 35 Alex Williamson 2014-05-02 05:50:59 UTC
I've proposed a patch series that should resolve this:

https://lkml.org/lkml/2014/5/1/290

The patches applied to v3.15-rc3 can also be found here:

git://github.com/awilliam/linux-vfio.git dma-alias

Please test and provide feedback here or to the list.  This should resolve the warning in all cases, but we may still need to tackle the requester ID used by these bridges.  Please report any IOMMU faults reported when trying to use devices behind the bridge.  Thanks
Comment 36 Daniel Gottschalk 2014-05-02 06:50:32 UTC
I've applied the patch and it works. Do you need dmesg output or further information?

Thank you!
Comment 37 Andrey Nazarov 2014-05-02 11:17:01 UTC
I get this on Supermicro X10SAE using current dma-alias branch:

[    0.797077] dmar: DRHD: handling fault status reg 3
[    0.797085] firewire_ohci 0000:04:03.0: added OHCI v1.10 device as card 0, 4 IR + 8 IT contexts, quirks 0x2
[    0.797159] dmar: DMAR:[DMA Read] Request device [04:00.0] fault addr fffff000 
[    0.797159] DMAR:[fault reason 02] Present bit in context entry is clear
[    0.797198] firewire_ohci 0000:04:03.0: bad self ID 0/1 (00000000 != ~00000000)
Comment 38 Alex Williamson 2014-05-02 15:00:55 UTC
(In reply to Andrey Nazarov from comment #37)
> I get this on Supermicro X10SAE using current dma-alias branch:
> 
> [    0.797077] dmar: DRHD: handling fault status reg 3
> [    0.797085] firewire_ohci 0000:04:03.0: added OHCI v1.10 device as card
> 0, 4 IR + 8 IT contexts, quirks 0x2
> [    0.797159] dmar: DMAR:[DMA Read] Request device [04:00.0] fault addr
> fffff000 
> [    0.797159] DMAR:[fault reason 02] Present bit in context entry is clear
> [    0.797198] firewire_ohci 0000:04:03.0: bad self ID 0/1 (00000000 !=
> ~00000000)

Andrey, please try the dma-alias-test branch, I've added a couple patches to quirk bridges that behave a PCIe-to-PCI bridges despite lack of a PCIe capability.
Comment 39 Alex Williamson 2014-05-02 15:32:02 UTC
(In reply to Daniel Gottschalk from comment #36)
> I've applied the patch and it works. Do you need dmesg output or further
> information?
> 
> Thank you!

Hi Daniel, I don't see that we have lspci info for your system recorded in the bz, could you provide 'sudo lspci -v'.  Thanks
Comment 40 Daniel Gottschalk 2014-05-02 19:34:57 UTC
Hi Alex,

here is the lspci -v output:

00:00.0 Host bridge: Intel Corporation Haswell DRAM Controller (rev 06)
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, fast devsel, latency 0
	Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:01.0 PCI bridge: Intel Corporation Haswell PCI Express x16 Controller (rev 06) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	Capabilities: [88] Subsystem: ASUSTeK Computer Inc. Device 8534
	Capabilities: [80] Power Management version 3
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [a0] Express Root Port (Slot+), MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [140] Root Complex Link
	Capabilities: [d94] #19
	Kernel driver in use: pcieport

00:01.1 PCI bridge: Intel Corporation Haswell PCI Express x8 Controller (rev 06) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: f7c00000-f7cfffff
	Prefetchable memory behind bridge: 00000000f0000000-00000000f1ffffff
	Capabilities: [88] Subsystem: ASUSTeK Computer Inc. Device 8534
	Capabilities: [80] Power Management version 3
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [a0] Express Root Port (Slot+), MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [140] Root Complex Link
	Capabilities: [d94] #19
	Kernel driver in use: pcieport

00:02.0 VGA compatible controller: Intel Corporation Haswell Integrated Graphics Controller (rev 06) (prog-if 00 [VGA controller])
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, fast devsel, latency 0, IRQ 44
	Memory at f7400000 (64-bit, non-prefetchable) [size=4M]
	Memory at e0000000 (64-bit, prefetchable) [size=256M]
	I/O ports at f000 [size=64]
	Expansion ROM at <unassigned> [disabled]
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [d0] Power Management version 2
	Capabilities: [a4] PCI Advanced Features
	Kernel driver in use: i915

00:03.0 Audio device: Intel Corporation Haswell HD Audio Controller (rev 06)
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, fast devsel, latency 0, IRQ 53
	Memory at f7d10000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 2
	Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: snd_hda_intel

00:14.0 USB controller: Intel Corporation Lynx Point USB xHCI Host Controller (rev 05) (prog-if 30 [XHCI])
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, medium devsel, latency 0, IRQ 52
	Memory at f7d00000 (64-bit, non-prefetchable) [size=64K]
	Capabilities: [70] Power Management version 2
	Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
	Kernel driver in use: xhci_hcd

00:16.0 Communication controller: Intel Corporation Lynx Point MEI Controller #1 (rev 04)
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at f7d1a000 (64-bit, non-prefetchable) [size=16]
	Capabilities: [50] Power Management version 3
	Capabilities: [8c] MSI: Enable- Count=1/1 Maskable- 64bit+

00:1a.0 USB controller: Intel Corporation Lynx Point USB Enhanced Host Controller #2 (rev 05) (prog-if 20 [EHCI])
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, medium devsel, latency 0, IRQ 20
	Memory at f7d18000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
	Kernel driver in use: ehci-pci

00:1c.0 PCI bridge: Intel Corporation Lynx Point PCI Express Root Port #1 (rev d5) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: f7b00000-f7bfffff
	Capabilities: [40] Express Root Port (Slot+), MSI 00
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8534
	Capabilities: [a0] Power Management version 3
	Kernel driver in use: pcieport

00:1c.1 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5) (prog-if 01 [Subtractive decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=04, subordinate=05, sec-latency=0
	I/O behind bridge: 0000c000-0000cfff
	Memory behind bridge: f7a00000-f7afffff
	Capabilities: [40] Express Root Port (Slot+), MSI 00
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8534
	Capabilities: [a0] Power Management version 3

00:1c.2 PCI bridge: Intel Corporation Lynx Point PCI Express Root Port #3 (rev d5) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
	I/O behind bridge: 0000b000-0000bfff
	Memory behind bridge: f7900000-f79fffff
	Capabilities: [40] Express Root Port (Slot+), MSI 00
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8534
	Capabilities: [a0] Power Management version 3
	Kernel driver in use: pcieport

00:1c.3 PCI bridge: Intel Corporation Lynx Point PCI Express Root Port #4 (rev d5) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=07, subordinate=07, sec-latency=0
	I/O behind bridge: 0000a000-0000afff
	Memory behind bridge: f7800000-f78fffff
	Capabilities: [40] Express Root Port (Slot+), MSI 00
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8534
	Capabilities: [a0] Power Management version 3
	Kernel driver in use: pcieport

00:1d.0 USB controller: Intel Corporation Lynx Point USB Enhanced Host Controller #1 (rev 05) (prog-if 20 [EHCI])
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, medium devsel, latency 0, IRQ 23
	Memory at f7d17000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
	Kernel driver in use: ehci-pci

00:1f.0 ISA bridge: Intel Corporation Lynx Point LPC Controller (rev 05)
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, medium devsel, latency 0
	Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:1f.2 SATA controller: Intel Corporation Lynx Point 6-port SATA Controller 1 [AHCI mode] (rev 05) (prog-if 01 [AHCI 1.0])
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 45
	I/O ports at f0b0 [size=8]
	I/O ports at f0a0 [size=4]
	I/O ports at f090 [size=8]
	I/O ports at f080 [size=4]
	I/O ports at f060 [size=32]
	Memory at f7d16000 (32-bit, non-prefetchable) [size=2K]
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [70] Power Management version 3
	Capabilities: [a8] SATA HBA v1.0
	Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation Lynx Point SMBus Controller (rev 05)
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: medium devsel, IRQ 18
	Memory at f7d15000 (64-bit, non-prefetchable) [size=256]
	I/O ports at f040 [size=32]

02:00.0 RAID bus controller: 3ware Inc 9690SA SAS/SATA-II RAID PCIe (rev 01)
	Subsystem: 3ware Inc 9690SA SAS/SATA-II RAID PCIe
	Flags: bus master, fast devsel, latency 0, IRQ 17
	Memory at f0000000 (64-bit, prefetchable) [size=32M]
	Memory at f7c20000 (64-bit, non-prefetchable) [size=4K]
	I/O ports at e000 [size=256]
	Expansion ROM at f7c00000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 2
	Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
	Capabilities: [70] Express Legacy Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Kernel driver in use: 3w-9xxx

03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
	Subsystem: Intel Corporation PRO/1000 PT Desktop Adapter
	Flags: bus master, fast devsel, latency 0, IRQ 51
	Memory at f7b40000 (32-bit, non-prefetchable) [size=128K]
	Memory at f7b20000 (32-bit, non-prefetchable) [size=128K]
	I/O ports at d000 [size=32]
	Expansion ROM at f7b00000 [disabled] [size=128K]
	Capabilities: [c8] Power Management version 2
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [e0] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-04-a5-60
	Kernel driver in use: e1000e

04:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03) (prog-if 01 [Subtractive decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=04, secondary=05, subordinate=05, sec-latency=32
	I/O behind bridge: 0000c000-0000cfff
	Memory behind bridge: f7a00000-f7afffff
	Capabilities: [c0] Subsystem: ASUSTeK Computer Inc. Device 8489

05:01.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
	Subsystem: Technotrend Systemtechnik GmbH Device 101a
	Flags: bus master, medium devsel, latency 32, IRQ 18
	Memory at f7a01000 (32-bit, non-prefetchable) [size=512]
	Kernel driver in use: budget_ci dvb

06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
	Subsystem: ASUSTeK Computer Inc. Device 8557
	Flags: bus master, fast devsel, latency 0, IRQ 18
	Memory at f7900000 (32-bit, non-prefetchable) [size=512K]
	I/O ports at b000 [size=32]
	Memory at f7980000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
	Capabilities: [a0] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Device Serial Number ac-22-0b-ff-ff-8c-19-92
	Capabilities: [1a0] Transaction Processing Hints
	Kernel driver in use: igb

07:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
	Subsystem: ASUSTeK Computer Inc. Device 8557
	Flags: bus master, fast devsel, latency 0, IRQ 21
	Memory at f7800000 (32-bit, non-prefetchable) [size=512K]
	I/O ports at a000 [size=32]
	Memory at f7880000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
	Capabilities: [a0] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Device Serial Number ac-22-0b-ff-ff-8c-19-93
	Capabilities: [1a0] Transaction Processing Hints
	Kernel driver in use: igb




Thank you!
Comment 41 Andrey Nazarov 2014-05-02 19:43:45 UTC
(In reply to Alex Williamson from comment #38)
> Andrey, please try the dma-alias-test branch, I've added a couple patches to
> quirk bridges that behave a PCIe-to-PCI bridges despite lack of a PCIe
> capability.

Thanks Alex. With the latest patches the IOMMU fault related to the built-in firewire controller is gone. I've also tried installing a NIC into one of the PCI slots on the motherboard and it properly worked with IOMMU enabled.
Comment 42 Alex Williamson 2014-05-02 19:54:46 UTC
(In reply to Daniel Gottschalk from comment #40)
> 04:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge
> (rev 03) (prog-if 01 [Subtractive decode])
>       Flags: bus master, fast devsel, latency 0
>       Bus: primary=04, secondary=05, subordinate=05, sec-latency=32
>       I/O behind bridge: 0000c000-0000cfff
>       Memory behind bridge: f7a00000-f7afffff
>       Capabilities: [c0] Subsystem: ASUSTeK Computer Inc. Device 8489
> 
> 05:01.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
>       Subsystem: Technotrend Systemtechnik GmbH Device 101a
>       Flags: bus master, medium devsel, latency 32, IRQ 18
>       Memory at f7a01000 (32-bit, non-prefetchable) [size=512]
>       Kernel driver in use: budget_ci dvb

Hi Daniel,

Thanks for the listing.  The bridge above is the device that was causing problems, have you attempted to use the dvb card behind it with these patches yet?  If not, please attempt to use it with the dma-alias branch or upstream patch series with VT-d enabled and report the results.  Thanks
Comment 43 Alex Williamson 2014-05-02 19:56:45 UTC
(In reply to Andrey Nazarov from comment #41)
> (In reply to Alex Williamson from comment #38)
> > Andrey, please try the dma-alias-test branch, I've added a couple patches
> to
> > quirk bridges that behave a PCIe-to-PCI bridges despite lack of a PCIe
> > capability.
> 
> Thanks Alex. With the latest patches the IOMMU fault related to the built-in
> firewire controller is gone. I've also tried installing a NIC into one of
> the PCI slots on the motherboard and it properly worked with IOMMU enabled.

Great, that means that the Tundra bridge needs to be handled as a PCIe-to-PCI bridge even though it doesn't have a PCIe capability.  Thanks for the test.
Comment 44 Daniel Gottschalk 2014-05-03 10:29:07 UTC
Hi Alex,

I did'nt check the dvb card. It does not work with these patches. So I tried the dma-alias branch, but the card doesn't work.

I saw the following messages in dmesg:

May  3 12:24:31 server kernel: [   57.332542] dmar: DRHD: handling fault status reg 3
May  3 12:24:31 server kernel: [   57.342311] dmar: DMAR:[DMA Read] Request device [05:00.0] fault addr ffffc000 
May  3 12:24:31 server kernel: [   57.342311] DMAR:[fault reason 02] Present bit in context entry is clear
May  3 12:24:31 server kernel: [   57.382378] dmar: DRHD: handling fault status reg 3
May  3 12:24:31 server kernel: [   57.392130] dmar: DMAR:[DMA Write] Request device [05:00.0] fault addr fffff000 
May  3 12:24:31 server kernel: [   57.392130] DMAR:[fault reason 02] Present bit in context entry is clear
May  3 12:24:31 server kernel: [   57.420280] dmar: DRHD: handling fault status reg 2
May  3 12:24:31 server kernel: [   57.430037] dmar: DMAR:[DMA Write] Request device [05:00.0] fault addr fffff000 
May  3 12:24:31 server kernel: [   57.430037] DMAR:[fault reason 02] Present bit in context entry is clear
May  3 12:24:31 server kernel: [   57.430047] dmar: DRHD: handling fault status reg 2
May  3 12:24:31 server kernel: [   57.430054] dmar: DMAR:[DMA Write] Request device [05:00.0] fault addr fffff000 
May  3 12:24:31 server kernel: [   57.430054] DMAR:[fault reason 02] Present bit in context entry is clear
May  3 12:24:31 server kernel: [   57.430061] dmar: DRHD: handling fault status reg 2
May  3 12:24:31 server kernel: [   57.430067] dmar: DMAR:[DMA Write] Request device [05:00.0] fault addr fffff000 
May  3 12:24:31 server kernel: [   57.430067] DMAR:[fault reason 02] Present bit in context entry is clear
May  3 12:24:31 server kernel: [   57.430080] dmar: DMAR:[DMA Write] Request device [05:00.0] fault addr fffff000 
May  3 12:24:31 server kernel: [   57.430080] DMAR:[fault reason 02] Present bit in context entry is clear

Regards, Daniel
Comment 45 Alex Williamson 2014-05-03 13:10:13 UTC
(In reply to Daniel Gottschalk from comment #44)
> Hi Alex,
> 
> I did'nt check the dvb card. It does not work with these patches. So I tried
> the dma-alias branch, but the card doesn't work.
> 
> I saw the following messages in dmesg:
> 
> May  3 12:24:31 server kernel: [   57.332542] dmar: DRHD: handling fault
> status reg 3
> May  3 12:24:31 server kernel: [   57.342311] dmar: DMAR:[DMA Read] Request
> device [05:00.0] fault addr ffffc000 
> May  3 12:24:31 server kernel: [   57.342311] DMAR:[fault reason 02] Present
> bit in context entry is clear

Thanks Daniel, this is actually good news, because otherwise you'd have an ASM1083 bridge that behaves different from mine.  I have a dma-alias-test branch that includes this extra patch:

https://github.com/awilliam/linux-vfio/commit/548a54070f3639295093bda0df9315b2dc4db04c

With this, the dvb card should work.  Thanks
Comment 46 Daniel Gottschalk 2014-05-03 15:06:27 UTC
Thank you, Alex. The dvb cards works fine with the dma-alias-test branch.
Comment 47 Milos Kaurin 2014-06-18 19:12:15 UTC
Hi  Alex,

I'm hitting the same warning with these specs:

Mobo: Gigabyte Q87M-D2H, BIOS F7 01/17/2014 (latest bios)
CPU: Intel Core i7 4770
Ram: 32GB @ 1600Mhz
GPU: Gigabyte Nvidia GTX 770, 4GB slightly overclocked by Gigabyte
http://www.techpowerup.com/vgabios/148828/gigabyte-gtx770-4096-130729.html
OS: Arch
Kernel:  3.15.0-rc7 (your git repo. commit: fd49c81f080a997aad4b0e73541cd42772b6a772, dma-alias branch)

I'm passing through my GPU (and it's HDMI audio) via pci-stub. Here is the "linux" line from my grub config file:
linux /vmlinuz-3.15.0-rc3-MILOS-14486-gad47fca root=/dev/mapper/sys-root ro pci-stub.ids=10de:1184,10de:0e0a intel_iommu=on vfio_iommu_type nomodeset

# lspci -vvv
00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
        Subsystem: Gigabyte Technology Co., Ltd Device 5000
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
        Latency: 0
        Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 0000e000-0000efff
        Memory behind bridge: d0000000-db0fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [88] Subsystem: Gigabyte Technology Co., Ltd Device 5000
        Capabilities: [80] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee002d8  Data: 0000
        Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0
                        ExtTag- RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 256 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #2, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <256ns, L1 <8us
                        ClockPM- Surprise- LLActRep- BwNot+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt+
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Via WAKE# ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Via WAKE# ARIFwd-
                LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                        Status: NegoPending- InProgress-
        Capabilities: [140 v1] Root Complex Link
                Desc:   PortNumber=02 ComponentID=01 EltType=Config
                Link0:  Desc:   TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
                        Addr:   00000000fed19000
        Capabilities: [d94 v1] #19
        Kernel driver in use: pcieport
        Kernel modules: shpchp

00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06) (prog-if 00 [VGA controller])
        Subsystem: Gigabyte Technology Co., Ltd Device d000
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 46
        Region 0: Memory at db400000 (64-bit, non-prefetchable) [size=4M]
        Region 2: Memory at c0000000 (64-bit, prefetchable) [size=256M]
        Region 4: I/O ports at f000 [size=64]
        Expansion ROM at <unassigned> [disabled]
        Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee00018  Data: 0000
        Capabilities: [d0] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [a4] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: i915
        Kernel modules: i915

00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
        Subsystem: Intel Corporation Device 2010
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 11
        Region 0: Memory at db824000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
        Kernel modules: snd_hda_intel

00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device 1c3a
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 44
        Region 0: Memory at db830000 (64-bit, non-prefetchable) [size=16]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [8c] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee00338  Data: 0000
        Kernel driver in use: mei_me
        Kernel modules: mei_me

00:16.3 Serial controller: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller (rev 04) (prog-if 02 [16550])
        Subsystem: Gigabyte Technology Co., Ltd Device 1c3a
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin B routed to IRQ 19
        Region 0: I/O ports at f0e0 [size=8]
        Region 1: Memory at db82e000 (32-bit, non-prefetchable) [size=4K]
        Capabilities: [c8] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Kernel driver in use: serial

00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device e000
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 45
        Region 0: Memory at db800000 (32-bit, non-prefetchable) [size=128K]
        Region 1: Memory at db82d000 (32-bit, non-prefetchable) [size=4K]
        Region 2: I/O ports at f080 [size=32]
        Capabilities: [c8] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee00378  Data: 0000
        Capabilities: [e0] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: e1000e
        Kernel modules: e1000e

00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 04) (prog-if 20 [EHCI])
        Subsystem: Gigabyte Technology Co., Ltd Device 5006
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at db82c000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [50] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Debug port: BAR=1 offset=00a0
        Capabilities: [98] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: ehci-pci
        Kernel modules: ehci_pci

00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device a002
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 11
        Region 0: Memory at db820000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
                        Status: NegoPending- InProgress-
                VC1:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=2 ArbSelect=Fixed TC/VC=04
                        Status: NegoPending- InProgress-
        Kernel modules: snd_hda_intel

00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d4) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
                        ClockPM- Surprise- LLActRep+ BwNot+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #0, PowerLimit 10.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
                        Changed: MRL- PresDet- LinkState-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR+, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [90] Subsystem: Gigabyte Technology Co., Ltd Device 5001
        Capabilities: [a0] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: pcieport
        Kernel modules: shpchp

00:1c.1 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d4) (prog-if 01 [Subtractive decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=03, subordinate=04, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #2, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
                        ClockPM- Surprise- LLActRep+ BwNot+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #1, PowerLimit 10.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR+, OBFF Not Supported ARIFwd-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [90] Subsystem: Gigabyte Technology Co., Ltd Device 5001
        Capabilities: [a0] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 04) (prog-if 20 [EHCI])
        Subsystem: Gigabyte Technology Co., Ltd Device 5006
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 23
        Region 0: Memory at db82b000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [50] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Debug port: BAR=1 offset=00a0
        Capabilities: [98] PCI Advanced Features
                AFCap: TP+ FLR+
                AFCtrl: FLR-
                AFStatus: TP-
        Kernel driver in use: ehci-pci
        Kernel modules: ehci_pci

00:1f.0 ISA bridge: Intel Corporation Q87 Express LPC Controller (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device 5001
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Capabilities: [e0] Vendor Specific Information: Len=0c <?>
        Kernel driver in use: lpc_ich
        Kernel modules: lpc_ich

00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 04) (prog-if 01 [AHCI 1.0])
        Subsystem: Gigabyte Technology Co., Ltd Device b005
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin B routed to IRQ 43
        Region 0: I/O ports at f0d0 [size=8]
        Region 1: I/O ports at f0c0 [size=4]
        Region 2: I/O ports at f0b0 [size=8]
        Region 3: I/O ports at f0a0 [size=4]
        Region 4: I/O ports at f060 [size=32]
        Region 5: Memory at db82a000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee00318  Data: 0000
        Capabilities: [70] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004
        Kernel driver in use: ahci
        Kernel modules: ahci

00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device 5001
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin C routed to IRQ 18
        Region 0: Memory at db829000 (64-bit, non-prefetchable) [size=256]
        Region 4: I/O ports at f040 [size=32]
        Kernel modules: i2c_i801

01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 770] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: Gigabyte Technology Co., Ltd Device 360c
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 11
        Region 0: Memory at da000000 (32-bit, non-prefetchable) [disabled] [size=16M]
        Region 1: Memory at d0000000 (64-bit, prefetchable) [disabled] [size=128M]
        Region 3: Memory at d8000000 (64-bit, prefetchable) [disabled] [size=32M]
        Region 5: I/O ports at e000 [disabled] [size=128]
        Expansion ROM at db000000 [disabled] [size=512K]
        Capabilities: [60] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [78] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <4us
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+
                         EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest+
        Capabilities: [b4] Vendor Specific Information: Len=14 <?>
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                        Status: NegoPending- InProgress-
        Capabilities: [128 v1] Power Budgeting <?>
        Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
        Capabilities: [900 v1] #19
        Kernel driver in use: pci-stub
        Kernel modules: nouveau

01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
        Subsystem: Gigabyte Technology Co., Ltd Device 360c
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin B routed to IRQ 10
        Region 0: Memory at db080000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: [60] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [78] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <4us
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+
                         EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest+
        Kernel driver in use: pci-stub
        Kernel modules: snd_hda_intel

03:00.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 41) (prog-if 01 [Subtractive decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=03, secondary=04, subordinate=04, sec-latency=32
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr+ DiscTmrStat- DiscTmrSERREn-
        Capabilities: [90] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=55mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [a0] Subsystem: Gigabyte Technology Co., Ltd Device 8892



Here is a Trace stack from my dmesg:

------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x65/0x90()
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.15.0-rc7-MILOS #1
Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./Q87M-D2H, BIOS F7 01/17/2014
 0000000000000000 00000000c9d5a959 ffff8808125e9d10 ffffffff81508e91
 0000000000000000 ffff8808125e9d48 ffffffff81068bad ffff880811dfa098
 ffff880811dfa000 ffff880811dfa098 0000000000000000 0000000000000000
Call Trace:
 [<ffffffff81508e91>] dump_stack+0x4d/0x6f
 [<ffffffff81068bad>] warn_slowpath_common+0x7d/0xa0
 [<ffffffff81068cda>] warn_slowpath_null+0x1a/0x20
 [<ffffffff812d1415>] pci_find_upstream_pcie_bridge+0x65/0x90
 [<ffffffff813ea0ad>] intel_iommu_add_device+0x4d/0x240
 [<ffffffff813df3a0>] ? bus_set_iommu+0x50/0x50
 [<ffffffff813df3ca>] add_iommu_group+0x2a/0x60
 [<ffffffff81390b53>] bus_for_each_dev+0x73/0xc0
 [<ffffffff813df398>] bus_set_iommu+0x48/0x50
 [<ffffffff8193c6c1>] intel_iommu_init+0x1bf/0x5d2
 [<ffffffff818ff297>] ? memblock_find_dma_reserve+0x143/0x143
 [<ffffffff818ff2a9>] pci_iommu_init+0x12/0x3c
 [<ffffffff81002162>] do_one_initcall+0xf2/0x1b0
 [<ffffffff8108a54d>] ? parse_args+0x21d/0x490
 [<ffffffff818f5067>] kernel_init_freeable+0x1a0/0x23c
 [<ffffffff814fde30>] ? rest_init+0x90/0x90
 [<ffffffff814fde3e>] kernel_init+0xe/0xf0
 [<ffffffff81516afc>] ret_from_fork+0x7c/0xb0
 [<ffffffff814fde30>] ? rest_init+0x90/0x90
---[ end trace d1fb1b388a28137f ]---


This also happens with the Arch kernel, which is why I tried your patche dma-alias.

I'm using this dma-alias commit: fd49c81f080a997aad4b0e73541cd42772b6a772

I've tried the dma-alias-test, but I've first hit the XHCI bug known in rc3 (dma-alias is rc7), and then my system would only boot to: "Net: registered protocol family 1"

My procedure for successfuly compiling dma-alias:
make mrproper
zcat /proc/config.gz > .config
make confold

and just edit the append-to-kernel-version to be something different then ARCH

This same procedure does not work for dma-alias-test on my system.

Dou you have any further recommendations on what to do next?
Comment 48 Alex Williamson 2014-06-18 19:38:33 UTC
(In reply to Milos Kaurin from comment #47)
> 00:1c.1 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d4) (prog-if 01
>         Bus: primary=00, secondary=03, subordinate=04, sec-latency=0
>         Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00

This is the bridge that's causing problems, it's attached under the above root port.

> 03:00.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 41) (prog-if 01

You'll want to try by dma-alias-v4 tag, with some slight modification.

Run 'lspci -n -s 3:00.0' and record the device ID, it's the number after 8086:

Then edit drivers/pci/quirks.c and add a line similar to:

DECLARE_PCI_FIXUP_HEADER(0x10e3, 0x8113, quirk_use_pcie_bridge_dma_alias);

on line 3409.  The new line will be the same except replacing 10e3 with 8086 and 8113 with the device ID you found above.
Comment 49 Milos Kaurin 2014-06-19 17:27:26 UTC
Hi,

This is what I've added:
DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias);

I no longer get the trace, and the kernel boots. Furthermore, attaching the GPU and it's HDMI sound device to vfio-pci does not give any errors:

This is from journalctl -b0 -ocat
/usr/bin/vfio-bind 0000:01:00.0 0000:01:00.1
VFIO - User Level meta-driver version: 0.3
vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

What else can I give you to make sure this is resolved in my case?

I've not yet been able to make GPU work, but this is because of my lack of KVM/QEMU, and this is not the place to discuss that :)

Thank you!
Comment 50 Alex Williamson 2014-06-19 20:50:07 UTC
(In reply to Milos Kaurin from comment #49)
> Hi,
> 
> This is what I've added:
> DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias);
> 
> I no longer get the trace, and the kernel boots. Furthermore, attaching the
> GPU and it's HDMI sound device to vfio-pci does not give any errors:
> 
> This is from journalctl -b0 -ocat
> /usr/bin/vfio-bind 0000:01:00.0 0000:01:00.1
> VFIO - User Level meta-driver version: 0.3
> vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
> vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
> vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
> vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
> vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
> vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
> 
> What else can I give you to make sure this is resolved in my case?
> 
> I've not yet been able to make GPU work, but this is because of my lack of
> KVM/QEMU, and this is not the place to discuss that :)

Hi Milos,

Do you have a conventional PCI (non-PCIe) device that you can install into that system and do a simple test to verify that it works with intel_iommu=on?  The code on the dma-alias-v4 tag will avoid the WARN_ON you hit previously regardless of the added line, but we can really only tell if we're handling DMA from behind that bridge correctly with a device installed and some simple testing.  Thanks
Comment 51 akamaka1024+bugtrackerkernelorg 2014-06-20 20:40:03 UTC
Hello,

I have an almost identical PC to Milos' and was experiencing this bug as well.
I compiled kernel 3.15.1 with this patch from tag dma-alias-v4 with the same extra line 
DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias); 
in drivers/pci/quirks.c and the error at boot-up stopped.

Additionally, to help with this annoying bug, I dug up an old Creative Audigy 2 PCI sound card and connected it. I blocked it from being picked up by linux with pci-stub and loaded qemu with it bound. After installing old Windows 7 drivers in Windows 8.1 Pro 64bit and a reboot, I can verify that sound worked fine. I did not do any heavy testing, I just checked that I was able to play some music.

lspci -vvv -nn:
00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5000]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: [e0] Vendor Specific Information: Len=0c <?>
	Kernel driver in use: hsw_uncore

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: d0000000-db0fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Gigabyte Technology Co., Ltd Device [1458:5000]
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00298  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0
			ExtTag- RBE+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 256 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <256ns, L1 <8us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt+
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet+ LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Via WAKE# ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Via WAKE# ARIFwd-
		LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
			 EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000
	Capabilities: [d94 v1] #19
	Kernel driver in use: pcieport
	Kernel modules: shpchp

00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller [8086:0412] (rev 06) (prog-if 00 [VGA controller])
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:d000]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 48
	Region 0: Memory at db400000 (64-bit, non-prefetchable) [size=4M]
	Region 2: Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Region 4: I/O ports at f000 [size=64]
	Expansion ROM at <unassigned> [disabled]
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00018  Data: 0000
	Capabilities: [d0] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [a4] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: i915
	Kernel modules: i915

00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06)
	Subsystem: Intel Corporation Device [8086:2010]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 49
	Region 0: Memory at db934000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee003f8  Data: 0000
	Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag- RBE-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 05) (prog-if 30 [XHCI])
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5007]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 43
	Region 0: Memory at db920000 (64-bit, non-prefetchable) [size=64K]
	Capabilities: [70] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
		Address: 00000000fee002f8  Data: 0000
	Kernel driver in use: xhci_hcd
	Kernel modules: xhci_hcd

00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:1c3a]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 45
	Region 0: Memory at db93f000 (64-bit, non-prefetchable) [size=16]
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [8c] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee00358  Data: 0000
	Kernel driver in use: mei_me
	Kernel modules: mei_me

00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection I217-LM [8086:153a] (rev 05)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:e000]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 47
	Region 0: Memory at db900000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at db93d000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at f080 [size=32]
	Capabilities: [c8] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee003d8  Data: 0000
	Capabilities: [e0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: e1000e
	Kernel modules: e1000e

00:1a.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 [8086:8c2d] (rev 05) (prog-if 20 [EHCI])
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5006]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at db93c000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci-pci
	Kernel modules: ehci_pci

00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 05)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:a002]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 46
	Region 0: Memory at db930000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee00398  Data: 0000
	Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag- RBE-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=01
			Status:	NegoPending- InProgress-
		VC1:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=2 ArbSelect=Fixed TC/VC=04
			Status:	NegoPending- InProgress-
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag- RBE+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
			ClockPM- Surprise- LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #0, PowerLimit 10.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: 00000000  Data: 0000
	Capabilities: [90] Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
	Capabilities: [a0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pcieport
	Kernel modules: shpchp

00:1c.1 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev d5) (prog-if 01 [Subtractive decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=03, subordinate=04, sec-latency=0
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: db800000-db8fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag- RBE+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
			ClockPM- Surprise- LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 10.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: 00000000  Data: 0000
	Capabilities: [90] Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
	Capabilities: [a0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 05) (prog-if 20 [EHCI])
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5006]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 23
	Region 0: Memory at db93b000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci-pci
	Kernel modules: ehci_pci

00:1f.0 ISA bridge [0601]: Intel Corporation Q87 Express LPC Controller [8086:8c4e] (rev 05)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: [e0] Vendor Specific Information: Len=0c <?>
	Kernel driver in use: lpc_ich
	Kernel modules: lpc_ich

00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05) (prog-if 01 [AHCI 1.0])
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:b005]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 44
	Region 0: I/O ports at f0d0 [size=8]
	Region 1: I/O ports at f0c0 [size=4]
	Region 2: I/O ports at f0b0 [size=8]
	Region 3: I/O ports at f0a0 [size=4]
	Region 4: I/O ports at f060 [size=32]
	Region 5: Memory at db93a000 (32-bit, non-prefetchable) [size=2K]
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00338  Data: 0000
	Capabilities: [70] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004
	Kernel driver in use: ahci
	Kernel modules: ahci

00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin C routed to IRQ 18
	Region 0: Memory at db939000 (64-bit, non-prefetchable) [size=256]
	Region 4: I/O ports at f040 [size=32]
	Kernel modules: i2c_i801

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104 [GeForce GTX 770] [10de:1184] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Micro-Star International Co., Ltd. Device [1462:2825]
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at da000000 (32-bit, non-prefetchable) [size=16M]
	Region 1: Memory at d0000000 (64-bit, prefetchable) [size=128M]
	Region 3: Memory at d8000000 (64-bit, prefetchable) [size=32M]
	Region 5: I/O ports at e000 [size=128]
	Expansion ROM at db000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+
			 EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest+
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [128 v1] Power Budgeting <?>
	Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Capabilities: [900 v1] #19
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau

01:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio Controller [10de:0e0a] (rev a1)
	Subsystem: Micro-Star International Co., Ltd. Device [1462:2825]
	Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 17
	Region 0: Memory at db080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+
			 EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest+
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

03:00.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev 41) (prog-if 01 [Subtractive decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=03, secondary=04, subordinate=04, sec-latency=32
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: db800000-db8fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr+ DiscTmrStat- DiscTmrSERREn-
	Capabilities: [90] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=55mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [a0] Subsystem: Gigabyte Technology Co., Ltd Device [1458:8892]

04:00.0 Multimedia audio controller [0401]: Creative Labs SB Audigy [1102:0004] (rev 04)
	Subsystem: Creative Labs SB Audigy2 Platinum [1102:1002]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 17
	Region 0: I/O ports at d000 [disabled] [size=64]
	Capabilities: [dc] Power Management version 2
		Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: vfio-pci
	Kernel modules: snd_emu10k1

04:00.1 Input device controller [0980]: Creative Labs SB Audigy Game Port [1102:7003] (rev 04)
	Subsystem: Creative Labs SB Audigy2 MIDI/Game Port [1102:0060]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Region 0: I/O ports at d040 [disabled] [size=8]
	Capabilities: [dc] Power Management version 2
		Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: vfio-pci
	Kernel modules: emu10k1_gp

04:00.2 FireWire (IEEE 1394) [0c00]: Creative Labs SB Audigy FireWire Port [1102:4001] (rev 04) (prog-if 10 [OHCI])
	Subsystem: Creative Labs SB Audigy FireWire Port [1102:0010]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 18
	Region 0: Memory at db804000 (32-bit, non-prefetchable) [disabled] [size=2K]
	Region 1: Memory at db800000 (32-bit, non-prefetchable) [disabled] [size=16K]
	Capabilities: [44] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: vfio-pci
	Kernel modules: firewire_ohci


qemu command line:

qemu-system-x86_64 \
-name Windows-8.1-Pro \
-enable-kvm -machine q35,accel=kvm \
-cpu host \
-m 8192 -smp 3,sockets=1,cores=3,threads=1 \
-bios /usr/share/qemu/bios.bin \
-balloon none \
-serial none \
-k en-gb \
-parallel none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device virtio-scsi-pci,id=scsi \
-drive file=/mnt/windows-8.1.img,id=disk,format=raw -device ide-hd,bus=ide.0,unit=0,drive=disk \
-boot menu=on \
-device vfio-pci,host=04:00.0,bus=root.1,addr=01.0,multifunction=on
-device vfio-pci,host=04:00.1,bus=root.1,addr=01.1
-device vfio-pci,host=04:00.2,bus=root.1,addr=01.2
-net none \
--monitor stdio \
-device qxl

after running qemu I got on dmesg:

Jun 20 23:04:24 XXX kernel: vfio-pci 0000:04:00.0: enabling device (0000 -> 0001)
Jun 20 23:04:24 XXX kernel: vfio-pci 0000:04:00.1: enabling device (0000 -> 0001)
Jun 20 23:04:24 XXX kernel: vfio-pci 0000:04:00.2: enabling device (0000 -> 0002)
Jun 20 23:04:35 XXX kernel: kvm: zapping shadow pages for mmio generation wraparound
Comment 52 Milos Kaurin 2014-06-23 13:10:36 UTC
Hi again.

To answer comment 50:
https://bugzilla.kernel.org/show_bug.cgi?id=44881#c50

I've purchased a PCI 10/100 realtek-based NIC for the purpose of testing.

I'd say it works fine with your patch. I have not tested without your patch (Would it even be able to attach it using vfio-pci ? )

My steps:

# unload the driver for the test NIC
rmmod 8139too

# prepare for passthrough
vfio-bind 0000:04:00.0

# start virt
qemu-system-x86_64 \
-name win7 \
-enable-kvm \
-M q35 \
-net bridge,br=br0 \
-net nic,macaddr=52:54:04:fa:8d:ee,model=virtio \
-nographic \
-spice addr=127.0.0.1,port=5900,disable-ticketing \
-vga qxl \
-cpu host \
-smp cores=2,threads=2 \
-m 8G \
-soundhw hda \
-balloon none \
-rtc base=localtime,clock=vm \
-drive file=/dev/sdb,index=0,if=virtio,discard=on \
-device vfio-pci,host=04:00.0

After connecting via spice, I could see the Realtek8139 card in device manager.
I've then logged out of spice, and moved my ethernet cable from the integrated NIC to the test NIC

I've been able to RDP into the Windows box without issue, using the IP which the test NIC picked up via DHCP

Do you need further testing?
Comment 53 Alex Williamson 2014-06-23 19:53:22 UTC
(In reply to Milos Kaurin from comment #52)
> Hi again.
> 
> To answer comment 50:
> https://bugzilla.kernel.org/show_bug.cgi?id=44881#c50
> 
> I've purchased a PCI 10/100 realtek-based NIC for the purpose of testing.
> 
> I'd say it works fine with your patch. I have not tested without your patch
> (Would it even be able to attach it using vfio-pci ? )

Hi Milos,

Could you clarify whether this testing was done with the dma-alias-v4 tag alone, or also with the additional quirk entry reported in comment 49.  There's really no need to do vfio-pci device assignment for testing, configuring the NIC for use in the host should be sufficient.  I don't think there's a need to test stock upstream, we know it's broken with this kind of bridge.  Thanks

(In reply to akamaka1024+bugtrackerkernelorg from comment #51)
> I have an almost identical PC to Milos' and was experiencing this bug as
> well.
> I compiled kernel 3.15.1 with this patch from tag dma-alias-v4 with the same
> extra line 
> DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias); 
> in drivers/pci/quirks.c and the error at boot-up stopped.
> 
> Additionally, to help with this annoying bug, I dug up an old Creative
> Audigy 2 PCI sound card and connected it. I blocked it from being picked up
> by linux with pci-stub and loaded qemu with it bound. After installing old
> Windows 7 drivers in Windows 8.1 Pro 64bit and a reboot, I can verify that
> sound worked fine. I did not do any heavy testing, I just checked that I was
> able to play some music.

Thanks for the testing.  I hope that Milos will report that his testing also included the quirk for 8086:244e.  Once confirmed I'll post a patch to add this officially.  Note that we're still waiting for the IOMMU subsystems and IOMMU driver changes to be accepted, so this additional quirk will not immediately fix things upstream.
Comment 54 Milos Kaurin 2014-06-23 20:22:01 UTC
Hi,

I've tested it with dma-alias-v4 + the quirk from comment 49.

Let me know if you need anything else
Comment 55 Alex Williamson 2014-06-23 22:37:04 UTC
Patch posted to enable 8086:244e
Comment 56 Andreas Sturmlechner 2014-07-04 17:53:21 UTC
Could this be backported to 3.14?
Comment 57 Alex Williamson 2014-07-04 21:38:06 UTC
It could be, but I won't be proposing it for -stable, it's far too large and intrusive for that.  Downstreams are of course welcome to backport.
Comment 58 Will 2014-10-26 11:31:04 UTC
I'm trying to pass a TDM410 phone card to an Asterisk guest using VT-d / IOMMU. I thought I may be effected by this bug and would appreciate some advice. Currently this seems to work ok in ubuntu kernel linux 3.2.0-70-generic but not when using 3.5.0-54-generic, 3.13.0-37-generic, 3.16.6-031606-generic, 3.17.1-031701-generic or 3.18.0-031800rc1-generic.

3.17.1 and 3.18.0-rc1 resolve the pci_find_upstream_pcie_bridge error message both still have the DMAR errors, the later has both read and write errors.

Motherboard: Gigabyte Technology Co., Ltd. Z87X-D3H/Z87X-D3H-CF, BIOS F9 08/25/2014 (latest)
03:00.0 Ethernet controller: Digium, Inc. Wildcard TDM410 4-port analog card (rev 11)
03:00.0 0200: d161:8005 (rev 11)

02:00.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 41) (prog-if 01 [Subtractive decode])
02:00.0 0604: 8086:244e (rev 41)

typical errors shown on the host when module loaded via modprobe in the guest....

Oct 26 06:35:39 kernel: [ 227.795652] dmar: DRHD: handling fault status reg 3
Oct 26 06:35:39 kernel: [ 227.795678] dmar: DMAR:[DMA Read] Request device [03:00.0] fault addr 73ac6000
Oct 26 06:35:39 kernel: [ 227.795678] DMAR:[fault reason 06] PTE Read access is not set

Modprobe freezes in the guest, guest is using high cpu and freezes on reboot possible spin lock. 

Just wondering if all the patches required to fix this are now present in the above kernels or if this could be a different problem?
Comment 59 Alan 2014-10-26 12:44:08 UTC
Please open a separate bug- the DMAR errors are almost certainly unrelated. Also given it seems to be an out of tree driver, please check that it correctly uses the DMA APIs throughout and that the vendor has tested it in this configuration.

Alan
Comment 60 Will 2014-10-26 13:35:42 UTC
(In reply to Alan from comment #59)
> Please open a separate bug- the DMAR errors are almost certainly unrelated.
> Also given it seems to be an out of tree driver, please check that it
> correctly uses the DMA APIs throughout and that the vendor has tested it in
> this configuration.
> 
> Alan

Ok thanks, will do.
Comment 61 Bjorn Helgaas 2015-03-09 22:08:03 UTC
The fix for this involves several commits that appeared in v3.17.