Created attachment 40772 [details]
3ware-9500s: lspci, dmesg, /proc/io{mem,ports}

The byte swap macros were never added to 3w-xxxx.  

Can you re-post 3w-9xxx lspci,dmesg, /proc/io{mem,port} attachment? 

The attachment doesn't actually contain what you reported it to.

-Adam

Created attachment 40782 [details]
3ware-9500s: lspci, dmesg, /proc/io{mem,ports}

Whoops. Sorry about that. Reattached.

Matt,

What slot do you have the card in, 32-bit or 64-bit?  Can you try the other
size slot?  I.E. 32->64 or 64->32.

This driver and card is known to work on ppc32 Linux and ppc64 Linux.  I did the big-endian driver changes and tested on my Apple G3 (32-bit) and G5 (64-bit).

Are you local to Silicon Valley?

-Adam

(In reply to comment #4)
> What slot do you have the card in, 32-bit or 64-bit?  Can you try the other
> size slot?  I.E. 32->64 or 64->32.

It's in a 64-bit slot, and the edge of the 32-bit slots is too thick for the card to fit, so no-can-do.

> Are you local to Silicon Valley?

No. :(

I'll cover twa_probe() with printks and report back tomorrow.

No need to add prints to twa_probe().  

In twa_poll_response(), just add this printk() and report the results:

response_queue.value = readl(TW_RESPONSE_QUEUE_REG_ADDR(tw_dev));

printk(KERN_WARNING "3w-9xxx: response_queue.value = 0x%x, request_id = 0x%x\n", response_queue.value, request_id);

response_request_id = TW_RESID_OUT(response_queue.response_id);
if (request_id != response_request_id) {
        TW_PRINTK(tw_dev->host, TW_DRIVER, 0x1e, "Found unexpected request id while polling for response");
        goto out;
}

We call twa_empty_response_queue() before this to junk garbage responses in the reigsters on power-up, so if we don't see anything even remotely close to the correct response in response_queue.value, that will tell us a lot.

-Adam

64-bit PCI slots on the Sibyte evaluation boards are implemented using a Hypertransport-to-PCI bridge.  Hypertransport however uses a separate port address space from the native 32-bit PCI slots that port x on 32-bit PCI is different from port x on the Hypertransport.  Memory mapped I/O is not affected by the issue.

The in/out functions don't know which of the busses / port address spaces they're operating on so they're always using the first bus that is the 32-bit PCI bus.  Note that the new style API with iomap() / ioreadX() / iowriteX() is not affected by this problem.

Would that explain the bug?

Ralf,

The 3w-9xxx driver is currently calling ioremap(), then using readl()/writel()
calls for memory mapped I/O.  

Are you suggesting we need to call iomap() and ioread32/iowrite32() for MIPS to work?

-Adam

No.

The issue I mentioned only exists for ioports, those little critters accessed with inX / outX instructions on x86.  readl / writel etc. are working ok afair.

And again it's only this multi-bus scenario that is affected.

  Ralf

Created attachment 40932 [details]
9500s debugging

Doesn't look bizarre to me.

I tried in booting the system in little-endian mode, and it crashes in the same way. The added printk statements print exactly what they print when in big-endian mode.

Matt,

You can prevent the crash from happening by commenting out or removing this line:
        
printk(KERN_WARNING "3w-9xxx: scsi%d: Firmware %s, BIOS %s, Ports: %d.\n",
        host->host_no,
        (char *)twa_get_param(tw_dev, 0, TW_VERSION_TABLE,
        TW_PARAM_FWVER, TW_PARAM_FWVER_LENGTH),
        (char *)twa_get_param(tw_dev, 1, TW_VERSION_TABLE,
        TW_PARAM_BIOSVER, TW_PARAM_BIOSVER_LENGTH),
        le32_to_cpu(*(int *)twa_get_param(tw_dev, 2, TW_INFORMATION_TABLE,
        TW_PARAM_PORTCOUNT, TW_PARAM_PORTCOUNT_LENGTH)));

I was a little over-zealous in assuming the controller would be responding here, or you would have already hung the system in POST.

At this point, since your card won't work in little-endian mode, I am guessing:

1. Your card is bad, which you could test by trying another 9500-S card.
2. The PCI slot you are trying is somehow incompatible with the 3ware hardware.

-Adam

I've confirmed that the card works in an AMD64 system (with a few unknown opcode messages in dmesg, but I suppose that's nothing particularly strange).

Ralf, do you have any ideas about why it wouldn't work with this board?

http://www.cyrius.com/debian/bcm91250a/hardware.html lists hardware that is confirmed working with the bcm91250a.

When I received it, it had installed a Silicon Image card, which I've confirmed doesn't work, much in the same way everything else doesn't work with recent kernels. I can only imagine that it used to work, since it was installed.

At the time, I think the system had 2.6.18 on it, so I guess I'll confirm that .18 works and then try to bisect.

Created attachment 60822 [details]
9500S debugging - 1GB RAM

Adam,

This board has a strange memory layout, which screws with PCI cards if you have more than 1GB of RAM, so the attached file is the debugging statement you requested with only 1GB of RAM installed (I had 2GB installed before). (In big-endian mode this time)

As you can see, there's no catastrophic null pointer dereference. Perhaps this is useful? Let me know, thanks.

I should have added that with 1GB of RAM, a 32-bit sata_sil card, 32-bit aic7xxx scsi card, and a 64-bit tg3 ethernet card work. With 2GB RAM, only the 64-bit tg3 ethernet card works.

The 3Ware 9500S is the only card I've tried that's failed with 1GB of RAM.

I gave up on the 3Ware 9500S and got a 64-bit PCI silicon image SATA controller that works fine.