Bug 43265

Summary: USB storage device capacity is detected incorrectly and the device is not writeable
Product: IO/Storage Reporter: Sturm Flut (sturmflut)
Component: Block LayerAssignee: Alan Stern (stern)
Status: CLOSED CODE_FIX    
Severity: high CC: alan, sergeyn, stern, sturmflut, towo, very.evil.odmin
Priority: P1    
Hardware: x86-64   
OS: Linux   
URL: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/997021
Kernel Version: 3.4.0-rc1 Subsystem:
Regression: Yes Bisected commit-id:
Attachments: dmesg ouput for kernel 3.4.0-rc6
Output of lsusb -v on kernel 3.4.0-rc6
Output of lspci -vv on kernel 3.4.0-rc6
dmesg output for kernel 3.2.0 (working)
usbmon capture for the non-working case
usbmon capture for the working case

Description Sturm Flut 2012-05-19 09:18:28 UTC
This was found to be an upstream regression, the original bug report can be found on https://bugs.launchpad.net/ubuntu/+source/linux/+bug/997021 .

I am having serious problems with an Intenso Rainbow Line 16 GB USB storage device (based on the "Silicon Motion 64MB QDI U2 DISK" chip, id 090c:1000) on my Hewlett-Packard HP Compaq 6710b (Intel Centrino, x86_64) notebook with development kernel 3.4.0-rc6.

On kernel 3.2.0 x86_64, older Linux kernels and Windows 7 the device works as expected. The capacity is reported correctly (16 GB) and the raw speed is within manufacturer specifications (~30 Mbyte/s read, 8 Mbyte/s write).

On kernel 3.4.0-rc6 x86_64 the device capacity is reported as 18 Exabytes (!) instead of 16 Gigabytes, and I can only read the device, but not write to it. Every write attempt ends in "Device not seekable" and/or "No space left on device":

sturmflut@cloud:~/tmp/bug$ sudo dd_rescue /dev/zero /dev/sdb
dd_rescue: (warning): output file is not seekable!
dd_rescue: (warning): Invalid argument
dd_rescue: (warning): Don't use sparse writes for non-seekable output
dd_rescue: (info): ipos: 0.0k, opos: 0.0k, xferd: 0.0k
                   errs: 0, errxfer: 0.0k, succxfer: 0.0k
             +curr.rate: 0kB/s, avg.rate: 0kB/s, avg.load: 0.0%

dd_rescue: (warning): write /dev/sdb (0.0k): No space left on device
dd_rescue: (warning): assumption rd(65536) == wr(0) failed!
dd_rescue: (fatal): write /dev/sdb (64.0k): No space left on device!

dd_rescue: (info): Summary for /dev/zero -> /dev/sdb:
dd_rescue: (info): ipos: 64.0k, opos: 64.0k, xferd: 64.0k
                   errs: 1, errxfer: 0.0k, succxfer: 0.0k
             +curr.rate: 178273kB/s, avg.rate: 153846kB/s, avg.load: 0.0%

sturmflut@cloud:~/tmp/bug$ sudo fdisk /dev/sdb
fdisk: unable to seek on /dev/sdb: Invalid argument


Other USB storage devices work as expected.
Comment 1 Sturm Flut 2012-05-19 09:19:08 UTC
Created attachment 73327 [details]
dmesg ouput for kernel 3.4.0-rc6
Comment 2 Sturm Flut 2012-05-19 09:19:29 UTC
Created attachment 73328 [details]
Output of lsusb -v on kernel 3.4.0-rc6
Comment 3 Sturm Flut 2012-05-19 09:20:39 UTC
Created attachment 73329 [details]
Output of lspci -vv on kernel 3.4.0-rc6
Comment 4 Alan 2012-05-21 14:30:34 UTC
Can you attach the dmesg for the working case ?
Comment 5 Sturm Flut 2012-05-21 15:52:02 UTC
Created attachment 73344 [details]
dmesg output for kernel 3.2.0 (working)
Comment 6 Sturm Flut 2012-05-21 15:52:22 UTC
Sure, here it is.
Comment 7 Sturm Flut 2012-06-09 22:07:09 UTC
I did a bunch of additional testing with the following kernels:

Ubuntu distribution kernel 3.2.0-22-generic: Works
Ubuntu distribution kernel 3.2.0-24-generic: Works

Vanilla kernel 3.3.7: Works
Vanilla kernel 3.3.8: Works

Vanilla kernel 3.4.0-rc6: Does not work
Vanilla kernel 3.4.0: Does not work
Vanilla kernel 3.4.1: Does not work

Vanilla kernel 3.5.0-rc1: Could not get this one to boot on my machine
Vanilla kernel 3.5.0-rc2: Does not work

The kernel messages are identical to the already uploaded ones. I used usbmon to capture the traffic on the USB bus for both cases, there were no other USB devices plugged in.
Comment 8 Sturm Flut 2012-06-09 22:07:42 UTC
Created attachment 73548 [details]
usbmon capture for the non-working case
Comment 9 Sturm Flut 2012-06-09 22:07:59 UTC
Created attachment 73549 [details]
usbmon capture for the working case
Comment 10 Sturm Flut 2012-06-11 13:55:03 UTC
I can now confirm that this bug was introduced with 3.4-rc1.
Comment 11 Sturm Flut 2012-06-13 11:38:31 UTC
I tried to bisect this error, but when git bisect checked out commit 0f4e68cf6e70fc219f219799c799a8a3e3c13100, the kernel would no longer boot. I don't know how to continue in this case. Here is my bisect log, in case anyone's interested:


git bisect start
# good: [c16fa4f2ad19908a47c63d8fa436a1178438c7e7] Linux 3.3
git bisect good c16fa4f2ad19908a47c63d8fa436a1178438c7e7
# bad: [dd775ae2549217d3ae09363e3edb305d0fa19928] Linux 3.4-rc1
git bisect bad dd775ae2549217d3ae09363e3edb305d0fa19928
# bad: [b2094ef840697bc8ca5d17a83b7e30fad5f1e9fa] Merge tag 'sound-3.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound
git bisect bad b2094ef840697bc8ca5d17a83b7e30fad5f1e9fa
# bad: [bbdb32cb5b73597386913d052165423b9d736145] Fix pppol2tp getsockname()
git bisect bad bbdb32cb5b73597386913d052165423b9d736145
# good: [377526578f2c343ea281a918b18ece1fca65005c] libertas: remove dump_survey implementation
git bisect good 377526578f2c343ea281a918b18ece1fca65005c
# good: [b4017c5368f992fb8fb3a2545a0977082c6664e4] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
git bisect good b4017c5368f992fb8fb3a2545a0977082c6664e4
Comment 12 Alan 2012-06-13 11:55:45 UTC
You can use "git bisect skip" to skip a bisection indicating that for some reason that particular one cannot be tested.

Alan
Comment 13 Alan 2012-06-13 12:03:02 UTC
If that doesn't work can you see btw if the behaviour change is caused by commit 09b6b51b0b6c1b9bb61815baf205e4d74c89ff04
Comment 14 Alan Stern 2012-06-13 14:41:45 UTC
I think the commit you identified is the correct one.  In fact, the problem should be fixed by a patch that was just submitted today:

    http://marc.info/?l=linux-usb&m=133958087430103&w=2
Comment 15 Sturm Flut 2012-06-13 21:17:24 UTC
I can confirm that the problem was introduced by 09b6b51b0b6c1b9bb61815baf205e4d74c89ff04 and is fixed by the patch submitted by Hans de Goede.

For some strange reason the bug did not show itself when plugging the mentioned device into a Sony notebook running stable kernel 3.4.2 today, although it should have been triggered.
Comment 16 Sturm Flut 2012-06-14 06:31:30 UTC
The patch was pulled into git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git and should show up in linux-next soon, effectively fixing the bug in kernel 3.5-rc3 or 3.5-rc4. This bug report may be closed.
Comment 17 Sergey 2012-06-15 17:51:19 UTC
Applied patch but it doesn't help

Here is an dmesg:

[  313.709127] usb 1-1.2: new high-speed USB device number 11 using ehci_hcd
[  313.795877] scsi10 : usb-storage 1-1.2:1.0
[  315.099497] scsi 10:0:0:0: Direct-Access     JetFlash Transcend 8GB    1100 PQ: 0 ANSI: 4
[  315.101734] sd 10:0:0:0: [sde] 71776119061217281 512-byte logical blocks: (18.3 EB/15.8 EiB)
[  315.102493] sd 10:0:0:0: [sde] Write Protect is off
[  315.102504] sd 10:0:0:0: [sde] Mode Sense: 43 00 00 00
[  315.103254] sd 10:0:0:0: [sde] No Caching mode page present
[  315.103265] sd 10:0:0:0: [sde] Assuming drive cache: write through
[  315.108117] sd 10:0:0:0: [sde] No Caching mode page present
[  315.108128] sd 10:0:0:0: [sde] Assuming drive cache: write through
[  315.110264]  sde: sde1
[  315.113350] sd 10:0:0:0: [sde] No Caching mode page present
[  315.113362] sd 10:0:0:0: [sde] Assuming drive cache: write through
[  315.113371] sd 10:0:0:0: [sde] Attached SCSI removable disk
[  315.277021] mount: sending ioctl 5310 to a partition!
[  315.277033] mount: sending ioctl 5310 to a partition!
[  315.278250] ISO 9660 Extensions: Microsoft Joliet Level 3
[  315.279616] ISO 9660 Extensions: RRIP_1991A

And kernel version:
Linux fix-fj 3.4.2-2-ARCH #1 SMP PREEMPT Fri Jun 15 21:32:27 MSK 2012 x86_64 GNU/Linux
Comment 18 Alan Stern 2012-06-15 18:07:25 UTC
There's no reason why the patch should help you.  The patch was written for Sturm's Feiya QDI U2 disk, but you have a Jetflash Transcend.
Comment 19 Sergey 2012-06-15 18:14:30 UTC
I don't understand. Does every flash drive should have different patch to work with 3.4+ kernel?
There is no fix for all flash drives?
Comment 20 Alan 2012-06-16 11:36:32 UTC
Almost no flash drives need fixes, because they follow the standards and work right
Comment 21 Sergey 2012-06-16 11:49:28 UTC
replacing
+UNUSUAL_DEV( 0x090c, 0x1000, 0x0000, 0xffff,
to
+UNUSUAL_DEV( 0x8564, 0x1000, 0x0000, 0xffff,

is workng for my flash drive
Comment 22 Alan Stern 2012-06-16 14:39:07 UTC
In fact, Sergey's bug report is the fourth one we've seen (although two of them are for the same device).  I have proposed adding a new flag for all USB mass-storage devices, to cause the sd driver to issue READ CAPACITY(10) before attempting READ CAPACITY(16).
Comment 23 Alan 2012-06-18 10:31:05 UTC
*** Bug 43391 has been marked as a duplicate of this bug. ***
Comment 24 Alan Stern 2012-07-23 18:37:56 UTC
This problem has been fixed by commit 6a0bdffa0073857870a4ed1b4489762146359eb4 (SCSI & usb-storage: add try_rc_10_first flag).
Comment 25 Alan Stern 2012-07-23 18:38:21 UTC
Closing out.