Bug 68951

Summary: Corrupt btrfs filesystem cannot be fixed by btrfs fsck or by mounting with -o recovery
Product: File System Reporter: Philip Armstrong (phil)
Component: btrfsAssignee: Josef Bacik (josef)
Status: RESOLVED OBSOLETE    
Severity: normal CC: alan
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 3.11.?, 3.12.6-2 Subsystem:
Regression: No Bisected commit-id:

Description Philip Armstrong 2014-01-18 16:06:56 UTC 
The btrfs filesystem on my laptop became corrupted during a suspend / resume cycle. Afterwards, kernel 3.11 could no longer mount the filesystem and btrfs fsck crashed with abort() errors. Not having much time to deal with it, I took an image and re-installed the laptop from backups.

I've just tried to mount a copy of the image with 3.12, and the kernel successfully mounts, but a find . inside the fiesystem, segfaults with the following error in the kernel log:

[ 5125.394350] BTRFS critical (device dm-3): unable to find logical 1885490642944 len 4096
[ 5125.394435] ------------[ cut here ]------------
[ 5125.394525] kernel BUG at /build/linux-xS3nxO/linux-3.12.6/fs/btrfs/inode.c:1595!
[ 5125.394649] invalid opcode: 0000 [#1] SMP 
[ 5125.394726] Modules linked in: ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp bridge stp llc ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables cpufreq_stats cpufreq_powersave cpufreq_conservative parport_pc cpufreq_userspace ppdev lp parport binfmt_misc deflate ctr twofish_generic twofish_x86_64_3way twofish_x86_64 twofish_common camellia_generic camellia_x86_64 serpent_sse2_x86_64 serpent_generic ablk_helper cryptd xts lrw gf128mul glue_helper blowfish_generic blowfish_x86_64 blowfish_common cast5_generic cast_common des_generic cbc cmac xcbc rmd160 sha512_generic sha256_generic hmac crypto_null af_key xfrm_algo snd_hda_codec_hdmi evdev kvm_amd radeon snd_hda_codec_realtek kvm snd_hda_intel snd_hda_codec serio_raw pcspkr snd_seq sp5100_tco snd_seq_device snd_hwdep edac_mce_amd snd_pcm edac_core i2c_piix4 snd_page_alloc k10temp snd_timer ttm drm_kms_helper snd drm soundcore i2c_algo_bit acpi_cpufreq i2c_core wmi processor thermal_sys button it87 hwmon_vid firewire_sbp2 fuse loop dm_crypt autofs4 ext4 crc16 mbcache jbd2 btrfs xor raid6_pq crc32c libcrc32c dm_mirror dm_region_hash dm_log dm_mod sd_mod crct10dif_generic crc_t10dif crct10dif_common hid_generic sg sr_mod cdrom ata_generic usbhid hid firewire_ohci firewire_core crc_itu_t ahci libahci pata_atiixp ehci_pci libata ohci_pci ohci_hcd ehci_hcd scsi_mod usbcore usb_common r8169 mii
[ 5125.397359] CPU: 1 PID: 16501 Comm: find Not tainted 3.12-1-amd64 #1 Debian 3.12.6-2
[ 5125.397485] Hardware name: Gigabyte Technology Co., Ltd. GA-MA78GPM-DS2H/GA-MA78GPM-DS2H, BIOS F6h 12/25/2010
[ 5125.397645] task: ffff8800aabe1040 ti: ffff880035d50000 task.ti: ffff880035d50000
[ 5125.397766] RIP: 0010:[<ffffffffa01f60fd>]  [<ffffffffa01f60fd>] btrfs_merge_bio_hook+0x6d/0x70 [btrfs]
[ 5125.397959] RSP: 0018:ffff880035d51a00  EFLAGS: 00010282
[ 5125.398047] RAX: 00000000ffffffea RBX: 0000000000001000 RCX: 0000000000000000
[ 5125.398163] RDX: ffff88012fc4ff98 RSI: ffff88012fc4e4a8 RDI: 0000000000000246
[ 5125.398278] RBP: 0000000000001000 R08: 000000000000000a R09: 000000000000044c
[ 5125.398393] R10: 0000000000000000 R11: ffff880035d515fe R12: 0000000000000000
[ 5125.398509] R13: 00000000db800008 R14: ffff88006337f628 R15: ffff88006b391af0
[ 5125.398626] FS:  00007f959502f700(0000) GS:ffff88012fc40000(0000) knlGS:0000000000000000
[ 5125.398756] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5125.398850] CR2: 00007fe350bb0000 CR3: 000000009cea9000 CR4: 00000000000007e0
[ 5125.398965] Stack:
[ 5125.399000]  0000000000001000 ffff880035d51c38 0000000000001000 ffffffffa020f56f
[ 5125.399136]  ffffea00014dbda0 0000000000000000 ffff8800cf53a080 0000002000001000
[ 5125.399270]  ffffffffa0210f70 0000000000000000 0000000000000000 000001b700001000
[ 5125.399404] Call Trace:
[ 5125.399486]  [<ffffffffa020f56f>] ? submit_extent_page.isra.38+0x17f/0x230 [btrfs]
[ 5125.399643]  [<ffffffffa0210f70>] ? repair_eb_io_failure+0xa0/0xa0 [btrfs]
[ 5125.399787]  [<ffffffffa020f885>] ? __do_readpage+0x265/0x780 [btrfs]
[ 5125.399923]  [<ffffffffa0210f70>] ? repair_eb_io_failure+0xa0/0xa0 [btrfs]
[ 5125.400065]  [<ffffffffa01ead90>] ? free_root_pointers+0x180/0x180 [btrfs]
[ 5125.400208]  [<ffffffffa020fe59>] ? __extent_read_full_page+0xb9/0xd0 [btrfs]
[ 5125.400352]  [<ffffffffa01ead90>] ? free_root_pointers+0x180/0x180 [btrfs]
[ 5125.400497]  [<ffffffffa02133d9>] ? read_extent_buffer_pages+0x209/0x2d0 [btrfs]
[ 5125.400645]  [<ffffffffa01ead90>] ? free_root_pointers+0x180/0x180 [btrfs]
[ 5125.400786]  [<ffffffffa01ebdf9>] ? btree_read_extent_buffer_pages.constprop.55+0xa9/0x110 [btrfs]
[ 5125.400960]  [<ffffffffa01ee62c>] ? read_tree_block+0x3c/0x70 [btrfs]
[ 5125.401089]  [<ffffffffa01cfce7>] ? read_block_for_search.isra.37+0x147/0x3a0 [btrfs]
[ 5125.401242]  [<ffffffffa01d5845>] ? btrfs_next_old_leaf+0x325/0x4b0 [btrfs]
[ 5125.401386]  [<ffffffffa01f77eb>] ? btrfs_real_readdir+0x13b/0x550 [btrfs]
[ 5125.401505]  [<ffffffff811850ae>] ? iterate_dir+0x9e/0xd0
[ 5125.401599]  [<ffffffff81063c50>] ? ns_capable+0x20/0x50
[ 5125.401690]  [<ffffffff8118549d>] ? SyS_getdents+0x8d/0x110
[ 5125.401785]  [<ffffffff811851b0>] ? fillonedir+0xd0/0xd0
[ 5125.401878]  [<ffffffff810d9a96>] ? __audit_syscall_exit+0x1f6/0x2a0
[ 5125.401986]  [<ffffffff81499b39>] ? system_call_fastpath+0x16/0x1b
[ 5125.402085] Code: 45 31 c0 89 fe 48 89 e1 48 89 c7 48 89 2c 24 e8 fa 50 02 00 85 c0 78 13 48 01 eb 31 c0 48 3b 1c 24 0f 97 c0 48 83 c4 08 5b 5d c3 <0f> 0b 90 f6 46 50 20 75 0a c3 66 0f 1f 84 00 00 00 00 00 53 48 
[ 5125.402638] RIP  [<ffffffffa01f60fd>] btrfs_merge_bio_hook+0x6d/0x70 [btrfs]
[ 5125.402786]  RSP <ffff880035d51a00>
[ 5125.445483] ---[ end trace f8deb58358aaba1a ]---

Meanwhile the latest git checkout of btrfs fsck dies as follows:

$ ./btrfsck /dev/mapper/Kantaka-Bill--dd
check_mounted(): Could not open /dev/mapper/Kantaka-Bill--dd
Could not check mount status: Permission denied
phil@xanthus ~/Code/btrfs/btrfs-progs (master)
$ sudo ./btrfsck /dev/mapper/Kantaka-Bill--dd
Checking filesystem on /dev/mapper/Kantaka-Bill--dd
UUID: a197b424-c577-49e3-855f-ccf0347e9c02
checking extents
leaf parent key incorrect 474267648
bad block 474267648
owner ref check failed [474267648 4096]
Errors found in extent allocation tree or chunk allocation
checking free space cache
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338428)
free space inode generation (0) did not match free space cache generation (338425)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338414)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338414)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338423)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (338424)
free space inode generation (0) did not match free space cache generation (337927)
checking fs roots
btrfsck: cmds-check.c:1320: walk_down_tree: Assertion `!(1)' failed.

I have a btrfs-image of the metadata, but it's 110Mb. Should I attach it, or make it available for download somewhere?

cheers, Phl
Comment 1 Philip Armstrong 2014-01-18 16:28:59 UTC 
NB. mount -o recovery doesn't appear to do anything, unless it's working away in the background?