Bug 96261

Summary: PCI core doesn't allow non-prefetchable PCIe BARs in prefetchable windows
Product: Drivers Reporter: Bjorn Helgaas (bjorn)
Component: PCIAssignee: drivers_pci (drivers_pci)
Status: NEW ---    
Severity: normal CC: benh
Priority: P1    
Hardware: All   
OS: Linux   
URL: http://lkml.kernel.org/r/CAE9FiQU1gJY1LYrxs+ma5LCTEEe4xmtjRG0aXJ9K_Tsu+m9Wuw@mail.gmail.com
Kernel Version: v4.0-rc1 Subsystem:
Regression: Yes Bisected commit-id:
Attachments: dmesg log (failing)
Yinghai proposed patches
dmesg log (working)

Description Bjorn Helgaas 2015-04-07 16:15:06 UTC
Created attachment 173301 [details]
dmesg log (failing)

David Ahern reported this on sparc T5-8:

  PCI: Claiming 0000:04:06.0: Resource 14: 0000800100000000..000080010fffffff [220c]
  PCI: Claiming 0000:05:00.0: Resource 0: 0000800100000000..0000800100001fff [204]
  pci 0000:05:00.0: can't claim BAR 0 [mem 0x800100000000-0x800100001fff]: no compatible bridge window

Firmware placed the non-prefetchable 05:00.0 BAR 0 in a prefetchable bridge
window (IORESOURCE_PREFETCH == 0x2000).  The PCI core thinks this is invalid.  The PCIe spec r3.0, sec 7.5.2.1, has an implementation note about why it *is* valid in certain cases.
Comment 1 Bjorn Helgaas 2015-04-07 16:15:59 UTC
Created attachment 173311 [details]
Yinghai proposed patches
Comment 2 Bjorn Helgaas 2015-04-07 16:17:09 UTC
Created attachment 173321 [details]
dmesg log (working)

Dmesg log with previous patches applied.
Comment 3 Benjamin Herrenschmidt 2015-04-08 03:59:05 UTC
Copy of a mail I sent on that subject. Those patches are wrong, please don't apply (or back them out):

<<<
These patches (from the above BZ) aren't right for one thing: You
shouldn't set the IORESOURCE_PREFETCH in the resource itself. This
*will* break stuff.

You can put the resource in a prefetchable region indeed, if you
are on a PCIe-only path (though we should have some arch flag
to check that the host bridge indeed doesn't do any prefetch,
on powerpc we are good there).

But you can't set the "prefetch" bit on the resource because that would
be losing the indication that this resource has side effects. This bit
is used in some cases to enable store gathering when mmap'ing via sysfs
and can be used for similar uses by drivers.

It's one thing to say "you can put non-prefetch BARs in prefetch windows
on that path", it's another one to completely make the BAR looks like
it's prefetchable when it's not.
>>>