Bug 57641

Summary: btrfs find-new is missing many files
Product: File System Reporter: g2p.code+kbugs
Component: btrfsAssignee: Josef Bacik (josef)
Status: RESOLVED CODE_FIX    
Severity: normal CC: dsterba, mithrandi
Priority: P1    
Hardware: All   
OS: Linux   
URL: http://comments.gmane.org/gmane.comp.file-systems.btrfs/25550
Kernel Version: 3.9 Subsystem:
Regression: Yes Bisected commit-id:

Description g2p.code+kbugs 2013-05-06 11:17:54 UTC
Happens in 3.9, but only on some filesystems.  Those filesystems happen to have extended inode refs enabled.
Comment 1 Josef Bacik 2013-05-06 13:13:00 UTC
Are the files that are missing happen to be ones with extended irefs?  I will try and reproduce locally.
Comment 2 g2p.code+kbugs 2013-05-06 14:12:18 UTC
I got one report from someone who didn't have extended inode refs enabled (https://github.com/g2p/bedup/issues/20#issuecomment-17476323), I've asked him to subscribe to this bug.

Most files are actually missing, on my filesystem there's a random mozilla checkout that contributes 13215 results, the rest (about 500) are random files from my home.  I haven't been able to reproduce with a clean filesystem.  I'll try to send debug-tree output but it seems like it's going to be very large.
Comment 3 Josef Bacik 2013-05-06 14:15:59 UTC
Instead of debug-tree can you do btrfs-image?  Pull from my git tree and use the btrfs-image in there

git://github.com/josefbacik/btrfs-progs.git

and then give me the steps you use to reproduce and tell me what I'm looking for so I can make sure I see exactly what you are seeing.
Comment 4 g2p.code+kbugs 2013-05-06 14:25:17 UTC
It failed to dump:

sudo ./btrfs-image -s -c6 /dev/mapper/blah blah.btrfs-image
checksum verify failed on 65536 found DA97CF61 wanted 6B
checksum verify failed on 65536 found DA97CF61 wanted 6B
Csum didn't match
Error reading metadata block
Error adding block -5
checksum verify failed on 65536 found DA97CF61 wanted 6B
checksum verify failed on 65536 found DA97CF61 wanted 6B
Csum didn't match
Error reading metadata block
Error flushing pending -5
Comment 5 g2p.code+kbugs 2013-05-06 14:26:35 UTC
Oh, I'll umount first.
Comment 6 g2p.code+kbugs 2013-05-06 14:30:19 UTC
Same error after unmounting.
Comment 7 David Sterba 2013-05-06 21:13:15 UTC
Please apply this patch to progs, fixes the checksum bug:

https://patchwork.kernel.org/patch/2526141/
Comment 8 g2p.code+kbugs 2013-05-06 21:43:15 UTC
Yup, turns out I didn't need to upload the image after all (it was a bit shy of 3GB).

I posted a fix for the search ioctl here:
http://comments.gmane.org/gmane.comp.file-systems.btrfs/25550
Comment 9 Josef Bacik 2013-05-07 01:43:59 UTC
Applied to btrfs-next, closing.  Thanks!