On power8 machines, we need some 64-bit prefetchable BARs be in 64-bit window. Although we supply such window, PCI core never allocates resource from it which leads to kernel panic, device failure and hindrance to the upcoming SRIOV implementation on this platform. Here is a solution proposed by Yinghai: http://patchwork.ozlabs.org/patch/328067/
Created attachment 132451 [details] Dmesg without Yinghai's patch Unrelated or sensitive messages are removed or masked
Created attachment 132461 [details] dmesg with Yinghai's patch
It would save me a lot of time if you would pull out a specific example of this happening, e.g., we restricted the prefetchable window of bridge X to be below 4GB, even though it is a 64-bit window, because of the 32-bit BAR of device Y.
(In reply to Bjorn Helgaas from comment #3) > It would save me a lot of time if you would pull out a specific example of > this happening, e.g., we restricted the prefetchable window of bridge X to > be below 4GB, even though it is a 64-bit window, because of the 32-bit BAR > of device Y. Domain 6 which just consists of a root port and EP will be a simple example. ("desmg without Yinghai's patch" is copied from console after kernel panic, I don't why many debug information is missing. You may need refer to "dmesg with Yinghai's patch" for window and BAR information) Domain 6's host bridge resource: > pci_bus 0006:00: root bus resource [mem 0x3d24800000000-0x3d2487ffeffff] (bus > address [0x80000000-0xfffeffff]) > pci_bus 0006:00: root bus resource [mem 0x3d24008000000-0x3d247ffffffff 64bit > pref] > pci_bus 0006:00: root bus resource [bus 00-ff] One EP (with 6 functions) is connected to root port. All functions have 64bit pref BARs and ROM BAR. BAR 15 of root port is restricted to be below 4G because of those ROM BARs. > pci 0006:00:00.0: disabling bridge window [mem 0x00000000-0xffffffffffffffff] > to [bus 01] (unused) > pci 0006:00:00.0: BAR 15: assigned [mem 0x3d24800000000-0x3d248017fffff pref] This is how it looks after Yinghai's fix: > pci 0006:00:00.0: BAR 15: assigned [mem 0x3d24008000000-0x3d2400fffffff 64bit > pref] > pci 0006:00:00.0: BAR 14: assigned [mem 0x3d24800000000-0x3d248007fffff]
This should be resolved by http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=5b28541552ef5eeffc41d6936105f38c2508e566 which will appear in v3.16-rc1.