Bug 16583

Summary:	ext4: delayed block allocation failed and application not respond any more
Product:	File System	Reporter:	Andrej Gelenberg (andrej.gelenberg)
Component:	ext4	Assignee:	fs_ext4 (fs_ext4)
Status:	RESOLVED OBSOLETE
Severity:	normal	CC:	alan, salecss, tytso
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	2.6.35.1	Subsystem:
Regression:	No	Bisected commit-id:

Description Andrej Gelenberg 2010-08-13 20:01:45 UTC

dmesg:

EXT4-fs (dm-6): warning: mounting fs with errors, running e2fsck is recommended
EXT4-fs (dm-6): recovery complete
EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: acl,commit=100
EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: acl,commit=100
...
EXT4-fs error (device dm-6): ext4_ext_search_left: inode #3510: (comm flush-254:6) ix (42117) != EXT_FIRST_INDEX (0) (depth 1)!
EXT4-fs (dm-6): delayed block allocation failed for inode 3510 at logical offset 42117 with max blocks 77 with error -5
This should not happen!!  Data will be lost

transmission-daemon (which had the data on dm-6) don't respond anymore and i can't kill it with kill -9. I can't shutdown the system too (only mit sys-rq keys).

Comment 1 Loredan Stancu 2012-05-24 13:13:00 UTC

Hi guys,

Any new on this? I just had the same problem with the ext4 file system after resizing it from a LVM.

I'm using RHEL 6.3, kernel 2.6.32-262.el6.x86_64.

What I did was to:

1. Umount the LVM partion
2. check it with fsck.ext4 tool (no errors reported)
3. resize the file system using resize2fs tool (no errors reported)
4. resize the LVM using lvresize tool (no error reported)
5. run fsck.ext4 for the resulted file system and no errors was reported.

I was able to mount the new file system but when trying to use db2 database from that partition I received the following file system error and db2 instance crashed suddenly:

EXT4-fs error (device dm-2): ext4_ext_search_left: inode #716: (comm db2sysc) ix (3714852) != EXT_FIRST_INDEX

A fix for me was to create a new LVM and rsync all the files from bad filesystem to the new one. And worked perfectly. A hadn't any problem to access the files from the bad file system.

I this a well known problem? 

Regards,
Loredan

Comment 2 Alan 2012-08-13 15:48:55 UTC

RHEL kernel bugs should be reported to Red Hat, the RHEL kernel differs siginficantly from upstream

Comment 3 Theodore Tso 2012-08-13 19:34:25 UTC

Also, in the original bug report, (which was against an usptream kernel, 2.6.35) note:

EXT4-fs (dm-6): warning: mounting fs with errors, running e2fsck is recommended

If this means the file system was mounted with inconsistencies noted in the past.

And sure enough...

EXT4-fs error (device dm-6): ext4_ext_search_left: inode #3510: (comm
flush-254:6) ix (42117) != EXT_FIRST_INDEX (0) (depth 1)!

... is a file system inconsistency. 

The second reporter, Loredan, was using a Red Hat modified 2.6.32 kernel, and the RHEL 2.6.32 kernel is significantly different from upstream.  I will note that it wasn't clear whether the file system was being grown or shrunk, but in the more typical case of growing the file system, in general you should resize the LVM first, and *then* run resize2fs to resize the filesystem to use the newly enlarged LV.

Regards,

-- Ted

Comment 4 Loredan Stancu 2012-08-14 07:35:39 UTC

Hi Ted,

Thank you for your reply.


What I wanted to do was to shrunk the files system. This is why first I run resize2fs (after unmounting and checking the file system with fsck.ext4).

Best Regards,
Loredan