Summary : copying file (apparently only the big ones) result in either *silent* data corruption, or SATA error (abnormal status). The Linux system can simply not be trusted and is unusable. Distribution: Debian (Sid) Hardware Environment: +-- HD Samsung SP2504C, 250G (sdc)| +-- Onboard SATA SIL 3112A + | md0 | +-- HD Samsung SP2504C, 250G (sdb)| | A7N8X Deluxe + | | +-- HD Seagate ST3250624NS 250G,sda +-- SitCOM C033 (SIL 3512) + | +-- DVDR Plextor PX 755 SA (scd0) | +-- IDE (nforce2) --- HD 30G Software Environment: no proprietary module loaded. On the SIL 3112A, the to disks are formated the same way : one big 249Gb partition and one small 980Kb swap partition. The two partitions are unified as a RAID 1 device, only using *linux soft raid*, no attempt to make use of the RAID capabilities of the SATA chip. The Seagate Disk is divided into a Windows partition and two FAT32 partitions used to exchange date between the two systems. By the way, the Windows system seems to work ok (not extensively tested yet). Steps to reproduce, with a *2.6.15 kernel *: 1/ silent corruption : Copy a big (3,9Gb) file from one place to another on the raid device. $ cp SAUVEGARDE/cstehlin.001.tar TEST7/ $ cmp SAUVEGARDE/cstehlin.001.tar TEST7/cstehlin.001.tar SAUVEGARDE/cstehlin.001.tar TEST7/cstehlin.001.tar sont diff
*** Bug 8120 has been marked as a duplicate of this bug. ***
*** Bug 8445 has been marked as a duplicate of this bug. ***
*** Bug 7257 has been marked as a duplicate of this bug. ***
*** Bug 7504 has been marked as a duplicate of this bug. ***
Summary: Affects 3112/4/3512/3132 Chipset: Nvidia There are two other closed cases, one VIA chipset and one Nvidia chipset which both indicate a BIOS upgrade fixed the problem (ie it was presumably chipset/hardware)
Andi said NVIDIA and AMD were trying to trying to track down a chipset-related corruption issue that affected sata_nv. I wonder if it is this same issue.
Does giving "iommu=soft" kernel parameter make any difference?
Cedric, Can you please test as suggested on #7. Thanks.
No change, still silent corruption. HTH hermes:~# uname -a && cat /proc/cmdline Linux hermes 2.6.21-2-k7 #1 SMP Mon Jun 25 21:37:29 UTC 2007 i686 GNU/Linux root=/dev/md0 ro iommu=soft hermes:~# ls -lh TEST0 -r-------- 1 root root 4,3G 2007-07-09 23:07 TEST0 hermes:~# cp TEST0 TEST1 && cmp TEST0 TEST1 TEST0 TEST1 sont différents: octet 5152325, ligne 20178
Created attachment 11990 [details] sata_sil-update-cache-line-programming.patch Please give a shot at the attached patch on top of 2.6.22. Thanks.
Dear all, in bug 5664 I described a similar problem with SATA Drive Samsung SP2504C and the SIL RAID controller. Until last weekend I used the old kernel mentioned in bug 5664 in oder to deal with the problem. Than I chanThanks to all of you for the help and good luck to you, Cedricged the main memory on the board and after that the data corruption stopped! This is more than surprising cause I have tested the main memory with memtest86+ more for over 10 hours more than once. Once again: memtest didn't report an error over all testing periods. Nvertheless changing from two Infineon 512MB to two 1GB Infineon/Quimoda modules solved all of my problems. Thanks to all of you for the help and good luck to you, Cedric! Ciao, Christian
*** Bug 5664 has been marked as a duplicate of this bug. ***
Cedric, can you try replacing your memory and see if this fixes your problem. It looks like the bug can be closed.
Sorry to say, but in my case (SIL3512A & Samsung HD501 500GB) patch or iommu=soft did not make any difference. I got same corruption problem with nvidia sata controller and tried iommu, this is different pc but same drive. Drive itself is 100% fine thru usb box.
Juhis99, can you explain your configuration more thoroughly? Are the 3512A and nv on the same machine?
The SIL3512A is a PCI addon card. Only one sata drive tested and that is samsung hd501 500gb, with and without 1.5G jumper. Drive works fine with sata usb mass storage box. PC #1: P3 with old BX chipset mb. Tested addon card 2.6.22 with iommu=soft and with the patch (not both at the same time, should I try?). This is the setup I would use SIL3512A and the Samsung drive. PC #2: Nvidia 430 chipset and AMD X2 3600+. Tested only 2.6.18 with and without iommu=soft. Any nvidia sw was not installed, this is clean installation of debian etch. Integrated nvidia sata-2 controller also does quiet corruption (no addon card installed). I have tried sata drive with SIL3512A addon card on this PC before, but the symptoms where same as on PC #1.
Can you test a different harddrive just in case?
Sorry, this is currently first and the only sata drive I own (with bad success story so far). I also suspected the drive in first place, but I have tried with PC #2 to install winxp, it works 100% thru both controllers (integrated nvidia and sil3512a addon). I currently use drive with usb2 mass storage box.
Created attachment 12200 [details] sata_sil-errata-update.patch Please give a shot at this patch.
hi folks, just reporting another user with a similar issue I also have the asus a7n8x with the onboard controller and 2 pci silicon image controllers: 01:09.0 RAID bus controller: Silicon Image, Inc. SiI 3114 [SATALink/SATARaid] Serial ATA Controller (rev 02) 01:0a.0 RAID bus controller: Silicon Image, Inc. SiI 3114 [SATALink/SATARaid] Serial ATA Controller (rev 02) 01:0b.0 RAID bus controller: Silicon Image, Inc. SiI 3112 [SATALink/SATARaid] Serial ATA Controller (rev 01) I've currently got nothing attached to the onboard controller, 1x320gb seagate sata drive, and 6x300gb sata [root@achilles array]# more /proc/scsi/scsi Attached devices: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: ST3320620AS Rev: 3.AA Type: Direct-Access ANSI SCSI revision: 05 Host: scsi2 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: ST3500630AS Rev: 3.AA Type: Direct-Access ANSI SCSI revision: 05 Host: scsi3 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: ST3500630AS Rev: 3.AA Type: Direct-Access ANSI SCSI revision: 05 Host: scsi4 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: ST3500630AS Rev: 3.AA Type: Direct-Access ANSI SCSI revision: 05 Host: scsi5 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: ST3500630AS Rev: 3.AA Type: Direct-Access ANSI SCSI revision: 05 Host: scsi6 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: ST3500630AS Rev: 3.AA Type: Direct-Access ANSI SCSI revision: 05 Host: scsi7 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: ST3500630AS Rev: 3.AA Type: Direct-Access ANSI SCSI revision: 05 [root@achilles array]# The single 320 is the drive i'm booting from, which I haven't seen corruption on yet. The 6 500gb drives are running with md in software raid and I've been getting corruption, on both raid 4 and 5 (I haven't tried 1 yet). EXT3-fs error (device md0): ext3_free_blocks_sb: bit already cleared for block 281015356 Aborting journal on device md0. ext3_abort called. EXT3-fs error (device md0): ext3_journal_start_sb: Detected aborted journal Remounting filesystem read-only EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_free_blocks_sb: Journal has aborted EXT3-fs error (device md0) in ext3_reserve_inode_write: Journal has aborted EXT3-fs error (device md0) in ext3_truncate: Journal has aborted EXT3-fs error (device md0) in ext3_reserve_inode_write: Journal has aborted EXT3-fs error (device md0) in ext3_orphan_del: Journal has aborted EXT3-fs error (device md0) in ext3_reserve_inode_write: Journal has aborted __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data __journal_remove_journal_head: freeing b_committed_data kjournald starting. Commit interval 5 seconds EXT3-fs warning (device md0): ext3_clear_journal_err: Filesystem error recorded from previous mount: IO failure EXT3-fs warning (device md0): ext3_clear_journal_err: Marking fs in need of filesystem check. EXT3-fs warning: mounting fs with errors, running e2fsck is recommended EXT3 FS on md0, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. Previously in this machine, I had 4x200gb seagate IDE drives (not sata) on a pci ide controller, and never had this problem with them - indeed, I've simply moved the old controller & drives to another machine, and it's still running. The goal was to migrate data to this array once I got it online (a process that started 2 months ago), but the data corruption issues have kept me from depending on it so far. I had orginally installed a 12port 3ware 9500s controller, but the high iowaits (another bug on here) & slow performance (max 22-24MB/s) on that moved me back to sw raid, which I can hit 100MB/s read & 70MB/s writes on. IN any case, I'll happily try some tests if you folks require. I've run this on both fedora core 6, 7, both on kernel 2.6.22.1, and currently back on centos with the distribution kernel 2.6.18-8.1.8.el5.
Oh, i should mention for clarification too, that again, nothing is attached to the onboard controller (sil 3112), all 7 drives are attached to the 2 pci sil 3114 controllers, with no raid configuration - all raid & drives are controlled by linux directly. dwight.
Dwight, please test patch posted in comment 19 on top of 2.6.22.1. Thanks.
patch applied [root@achilles linux]# patch < sata_sil-errata-update.patch can't find file to patch at input line 5 Perhaps you should have used the -p or --strip option? The text leading up to this was: -------------------------- |diff --git a/drivers/ata/sata_sil.c b/drivers/ata/sata_sil.c |index db67637..58f185a 100644 |--- a/drivers/ata/sata_sil.c |+++ b/drivers/ata/sata_sil.c -------------------------- File to patch: drivers/ata/sata_sil.c patching file drivers/ata/sata_sil.c Hunk #2 succeeded at 297 (offset -1 lines). Hunk #3 succeeded at 603 (offset -7 lines). [root@achilles linux]# I'll compile the kernel today and try it tonight. dwight.
Ok, I had to do some more kernal compiling, but I now have a 2.6.22-1 kernel compiled with the patch Tejun posted in comment #19. The 6 500gb sata seagate baracudas are mounted and the md0 volume, raid4, is running on them now. I've started some large file copies to it that I'll let run over night to see if the errors re-occur. Other than that, what's the best way to test it for issues - iozone or something similar? I had been previously running a windows vm guest on vmware server, but since installing the new kernel, i haven't been able to get my vmmon module recompiled. I'll fix that later. dwight.
Repeatedly copying large file to/from/inside raid and checksumming it should be a pretty good test.
Ok. iozone ran on the md0 volume all night, with a pretty consistent read/write of at least 10, at times 30-50MBps. So far these are the only ext3 errors in the messages file: Aug 3 00:53:11 achilles kernel: EXT3-fs: mounted filesystem with ordered data mode. Aug 3 01:01:26 achilles kernel: EXT3-fs error (device md0): ext3_free_blocks_sb: bit already cleared for block 516091799 Aug 3 01:01:26 achilles kernel: EXT3-fs error (device md0): ext3_free_blocks_sb: bit already cleared for block 516091800 Aug 3 01:01:26 achilles kernel: EXT3-fs error (device md0): ext3_free_blocks_sb: bit already cleared for block 516091801 Aug 3 01:01:26 achilles kernel: EXT3-fs error (device md0): ext3_free_blocks_sb: bit already cleared for block 516091802 Aug 3 04:02:09 achilles kernel: EXT3-fs error (device md0): htree_dirblock_to_tree: bad entry in directory #135168001: rec_len % 4 != 0 - offset=584, inode=1430531692, rec_len=19789, name_len=89 [root@achilles array]# I'll modify the iozone script i wrote to md5sum the files it creates every loop. I'm away all weekend, returning sunday, so i'll check on it again when i return. dwight.
Ok ... testing ran all weekend, which included running iozone , md5sum a large file, and then copying the file to another directory on the array: quick config overview 2 x sata_sil cards, firmware 3114 1 x 320gb seagate barracuda 6 x 500gb seagate barracuda (in software raid 4, mounted as /array/) asus a7n8x motherboard, 2gb memory centos 5, kernel 2.6.22.1, with sata_sil patch from Tejun iozone sh script (yes, it's not complex, but it works - i'm no programmer) ----- while (true) do sleep 2 DATE=`/bin/date +%G.%B.%d.%H:%M` OUTCOPY="/array/scratch/iozone/files/$DATE" mkdir $OUTCOPY /opt/iozone/bin/iozone -aR -g 4M -b output.wks -i 5 -i 6 -i 0 -w cp iozone.tmp $OUTCOPY /opt/iozone/bin/iozone -R -b output.wks -w -i 5 -i 6 -i 0 -t 40 -F \ $OUT/a $OUT/b $OUT/c $OUT/d $OUT/e $OUT/f $OUT/g $OUT/h $OUT/i $OUT/j $OUT/k $OUT/l $OUT/m $OUT/n $OUT/o $OUT/p $OUT/q $OUT/r $OUT/s $OUT/ t \ $OUT/aa $OUT/ab $OUT/ac $OUT/ad $OUT/ae $OUT/af $OUT/ag $OUT/ah $OUT/ai $OUT/aj $OUT/ak $OUT/al $OUT/am $OUT/an $OUT/ao $OUT/ap $OUT/aq $O UT/ar $OUT/as $OUT/at cp $OUT/* $OUTCOPY/ md5sum $OUTCOPY/* md5sum /array/storage/images/8gbfile cp /array/storage/images/8gbfile $OUTCOPY/ done --------- this script, i started running on friday at 1.16am, and it's still running. When I started, I was at about 1% used, now its at: /dev/md0 2.3T 1.3T 933G 58% /array ... 1.3TB used. mdadm details: [root@achilles iozone]# mdadm --detail /dev/md0 /dev/md0: Version : 00.90.03 Creation Time : Sun Jul 29 12:48:16 2007 Raid Level : raid4 Array Size : 2441918720 (2328.80 GiB 2500.52 GB) Device Size : 488383744 (465.76 GiB 500.10 GB) Raid Devices : 6 Total Devices : 6 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Mon Aug 6 00:20:04 2007 State : active Active Devices : 6 Working Devices : 6 Failed Devices : 0 Spare Devices : 0 Chunk Size : 256K UUID : 79e1543e:bec58433:69c2cb47:fbd4c005 Events : 0.5 Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1 2 8 49 2 active sync /dev/sdd1 3 8 65 3 active sync /dev/sde1 4 8 81 4 active sync /dev/sdf1 5 8 97 5 active sync /dev/sdg1 I'll attach another file in a minute, that is the output of "grep EXT3 /var/log/messages*" The volume is still online, and being written to/read from. Does this constitute a successful patch/test for you, Tejun? dwight.
Created attachment 12267 [details] ext3 errors from messages file After running tests for about 3 solid days (72 hours) this is the EXT3 errors from the kernel, into /var/log/messages. Filesystem is still mounted read-write and usage/testing continues - dwight.
ext3 error messages seem to indicate corruption. When I said 'copying & checksumming large files', I meant... 1. checksum the source file and store the checksum 2. copy the large file 3. checksum the copied file and compare with the checksum from #1. This way you can be sure data hasn't been corrupted during copy. Please fsck the filesystem and do the above test with a file way larger than your memory. Thanks.
ahhh, ok. I've reformatted the volume (ext3 again), rather than fsck, because its probably faster than trying to clear out 2TB worth of data. I've restarted the testing, with the following script. The file "/home/source/4gbfile" is another SATA drive, hopefully that shouldn't be an issue. ---------------------- #!/bin/sh OUT="/array/scratch/" SOURCE="/home/source/4gbfile" [root@achilles ~]# vi testarray.sh #!/bin/sh OUT="/array/scratch/" SOURCE="/home/source/4gbfile" while (true) do sleep 2 DATE=`/bin/date +%G.%B.%d.%H:%M` OUTCOPY="/array/scratch/$DATE" echo "run for $DATE" echo "copying file ..." mkdir -p $OUTCOPY cp $SOURCE $OUTCOPY echo "comparing file" echo $DATE | tee -a /home/source/compare.txt diff $SOURCE $OUTCOPY/4gbfile | tee -a /home/source/compare.txt echo "md5summing source and copy" md5sum $SOURCE >> $OUTCOPY/compare.txt md5sum $OUTCOPY/4gbfile >> $OUTCOPY/compare.txt done ----------------- I'll let it run for a day or 2 and let you know what it does. dwight.
well, after the first hour, every file copied has a different md5 checksum, and they themselves are different each time it is copied. I suspect this means i still have the issue? also, "diff" reports the file is different every time as well. Does this confirm for you that the issue still persists? Should I also try to copy a smaller file at the same time, and have it md5sum'ed as well? dwight.
Thanks, Dwight. That's enough. Unfortunately, I'm out of ideas at the moment. :-( I'll try to reproduce it here. Thanks.
Hmmm, darn. Well, I may try another hardware raid controller, anyone have suggestions for a recent, fast controller? The 3ware 9500s controller (pci-x in pci slot) i tried peaked out at about 22-24MB/s throughput, whereas the sw raid can do 70-100MBps. dwight s.
Latest patch on comment #19 didn't change anything, still cmp/diff errors. I also notice that it might be a read error; copied large file from working EIDE disk to SATA disk, compare it with cmp and got error, remount SATA and cmp gives different error position. Actually error position changes almost every remount of sata drive.
I unmounted/mounted, and md5sum'ed the file, and cmp'ed it to it's original. The md5's stayed the same, and all of the cmp differences were always in the same location. However, both my "/home/source/*" and "/array" are sata volumes. This machine doesn't have an internal IDE drive, but it does have an NFS mounted filesystem from another host I could copy from, if that would help. dwight. [root@achilles /]# umount /array ; mount /array ; md5sum /array/scratch/2007.August.06.13:11/4g* 8c16b5bf62628a3d0fcc8c031b33c979 /array/scratch/2007.August.06.13:11/4gbfile [root@achilles /]# umount /array ; mount /array ; md5sum /array/scratch/2007.August.06.13:11/4g* 8c16b5bf62628a3d0fcc8c031b33c979 /array/scratch/2007.August.06.13:11/4gbfile [root@achilles /]# umount /array ; mount /array ; md5sum /array/scratch/2007.August.06.13:11/4g* 8c16b5bf62628a3d0fcc8c031b33c979 /array/scratch/2007.August.06.13:11/4gbfile [root@achilles /]# umount /array ; mount /array ; md5sum /array/scratch/2007.August.06.13:11/4g* 8c16b5bf62628a3d0fcc8c031b33c979 /array/scratch/2007.August.06.13:11/4gbfile [root@achilles /]# umount /array ; mount /array ; md5sum /array/scratch/2007.August.06.13:11/4g* 8c16b5bf62628a3d0fcc8c031b33c979 /array/scratch/2007.August.06.13:11/4gbfile [root@achilles /]# umount /array ; mount /array ; md5sum /array/scratch/2007.August.06.13:11/4g* 8c16b5bf62628a3d0fcc8c031b33c979 /array/scratch/2007.August.06.13:11/4gbfile [root@achilles /]# cmp /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile differ: byte 10991085, line 41816 [root@achilles /]# umount /array ; mount /array ; cmp /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile differ: byte 10991085, line 41816 [root@achilles /]# umount /array ; mount /array ; cmp /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile differ: byte 10991085, line 41816 [root@achilles /]# umount /array ; mount /array ; cmp /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile differ: byte 10991085, line 41816 [root@achilles /]# umount /array ; mount /array ; cmp /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile differ: byte 10991085, line 41816 [root@achilles /]# umount /array ; mount /array ; cmp /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile differ: byte 10991085, line 41816 [root@achilles /]# umount /array ; mount /array ; cmp /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile /array/scratch/2007.August.06.13:11/4gbfile /home/source/4gbfile differ: byte 10991085, line 41816 [root@achilles /]#
Just a thought on this. Is this issue isolated to just this motherboard - the asus a7n8x? I'm a little surprised that an issue like this , data corruption, being present on SATA drives. Is this a widespread issue? If it's just this motherboard configuration, heck, i'll toss it and go to another board! Are other users complaining about SATA storage and data corruption? If it was widespread, i'm sure there be ALOT of noise from users. So, my system configuration is: amd 2800+ athlon xp asus a7n8x motherboard 2gb memory 2 x sata_sil 3114 sata controllers 6 x 500gb seagate sata drives centos version 5.0 kernel version 2.6.18, 2.6.22-1 Which of these components is most likely to be the issue? Again, if it was the sata drives / sata kernel driver, I think it would be more prevalent issue. if it's the sata_sil controllers, i'll try something else. Would it be the motherboard or cpu? if it's a hardware issue, I can try other hardware. I've been fighting this problem for about 3 months now, and i'm all for throwing hardware at it, if someone knows which of the components above is the issue. And pardon my ignorance, but are any of the folks watching this issue kernel developers? I know i'm not, i do good just to compile a kernel with no issues :) dwight.
AFAIK, the problem is sata_sil + nforce chipset combination. Dunno what's going on yet but the problem will go away if you change any one component of the combination. I'm planning on reproducing the problem here but haven't succeeded on finding the spare time. :-( I do a bit of kernel hacking and this problem is sorta well known among linux ATA folks. We just dunno how to fix it yet.
Ok, thanks for the feedback Tejun, much appreciated. I'll look into picking up an intel based board, cpu, and memory in the next few days, and test that out. I do have 3 sata_sil 4port controllers, 2 in my "main" system and a spare. I should be able to put the "old" parts into another case, attach a couple drives, and keep it online for further testing, whenever you want me to. dwight.
This is not only sata_sil (in my case pci card) + nvidia chipset. I can reproduce this with old intel bx chipset abit bh6 mainboard + addon sata_sil and also on asus M2NPV-VM which is nvidia chipset + addon sata_sil board. In my case same silent corruptation is with nvidia chipset integrated sata2 controller. (as said before, these both sata setups works fine on non-linux) My case could be somekind of compability with samsung hd501 500gb sata2 drive and linux drivers?
Not sure, but I have a new gigabit ds3 motherboard with the intel 956 chipset, intel 4300 cpu, and 2gb of new memory. I'll be swapping the sata_sil cards to this machine in the next day or two, and I'll let you know if the corruption continues. Tejun, should i just run the same test? ... cp file, md5 both the source & dest file after copying, cmp them, and see if they differ still? dwight.
Juhis, please give more details about your intel bx chipset + sata_sil setup. Nvidia integrated sata controller corruption is a different problem and will go away if you don't use iommu. I think recent kernels include workaround for this (disabling iommu by default). Dwight, copying a large file several times and md5summing them should be good enough. Thanks.
ok, I just got the system up and running, and copying data. On the first copy, they files were the same: [root@achilles ~]# more /array/scratch/2007.August.15.18\:17/compare.txt cad3e795c050446c654b19fde51b92c2 /home/source/4gbfile cad3e795c050446c654b19fde51b92c2 /array/scratch/2007.August.15.18:17/4gbfile [root@achilles ~]# lspci|grep -i sata 00:1f.2 SATA controller: Intel Corporation 82801HB (ICH8) 4 port SATA AHCI Controller (rev 02) 03:00.0 SATA controller: JMicron Technologies, Inc. JMicron 20360/20363 AHCI Controller (rev 02) 05:01.0 RAID bus controller: Silicon Image, Inc. SiI 3114 [SATALink/SATARaid] Serial ATA Controller (rev 02) 05:02.0 RAID bus controller: Silicon Image, Inc. SiI 3114 [SATALink/SATARaid] Serial ATA Controller (rev 02) [root@achilles ~]# I'll let it run for a few hours, and see if it stays consistent. If so, then i'm set. I'll keep the asus a7n8x here, and I'll get it setup if you want, Tejun, for further patch testing. How many drives should it need, just 2 - 1 on ide as a source, 2nd on sata_sil (3114 version) as a destination to test? dwight. ps - second copy was ok too: [root@achilles scratch]# cat 2007.August.15.18\:22/compare.txt cad3e795c050446c654b19fde51b92c2 /home/source/4gbfile cad3e795c050446c654b19fde51b92c2 /array/scratch/2007.August.15.18:22/4gbfile [root@achilles scratch]#
Ok, after about 25 copies, my md5's and cmp's are coming up the same on both source & destination files. So that solves the problem for me. Thanks for the help guys As I mentioned, I'll keep the old hardware handy and try to get it setup to do some testing on in the next couple weeks. Tejun (or anyone else making changes), just give me some warning if you want some testing done - I'll stay sub'ed to this bug so i get notices of updates & changes. dwight
*** Bug 2998 has been marked as a duplicate of this bug. ***
Any progress with this issue, does it still exist with recent kernel? One of the duplicate bugs had interesting comment: "The problem is disapearing when turning off software read-ahead (hdparm -a0 / dev/sdc). Unfortunately it brings 7x performance loss."
For me, after all the problem was on the disk itself, it took a long time before I believed it myself. After replacing the disk to identical there has not been problems. Sorry for the trouble!
Thanks for update. Closing the bug.