Bug 114281

Summary: df returns 100% used for empty mixed block group filesystem
Product: File System Reporter: Chris Murphy (bugzilla)
Component: btrfsAssignee: David Sterba (dsterba)
Status: RESOLVED PATCH_ALREADY_AVAILABLE    
Severity: normal CC: dsterba, luis, szg00000
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 4.5.0 Subsystem:
Regression: No Bisected commit-id:
Attachments: patch to fix the issue that made df return 0 available space
Patch for a related issue in the same code. Protect f_bfree from a potential overflow.

Description Chris Murphy 2016-03-10 08:22:25 UTC
Summary: A new empty mixed block group "-M" filesystem shows up as 100% full with regular df (gnu coreutils 8.24).

btrfs-progs-4.4.1-1.fc25.x86_64

Reproduce steps:


# mkfs.btrfs -dsingle -msingle b1.img	
# mkfs.btrfs -M -dsingle -msingle b2.mg
# losetup /dev/loop1 b1.img
# losetup /dev/loop2 b2.img
# df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/loop1      6817792    16576   6542272   1% /mnt/b1
/dev/loop2      6817792     4128         0 100% /mnt/b2

Umm, the 2nd one is clearly wrong. It's not full.


# btrfs fi usage /mnt/b1
Overall:
    Device size:		   6.50GiB
    Device allocated:		 276.00MiB
    Device unallocated:		   6.23GiB
    Device missing:		     0.00B
    Used:			 192.00KiB
    Free (estimated):		   6.24GiB	(min: 6.24GiB)
    Data ratio:			      1.00
    Metadata ratio:		      1.00
    Global reserve:		  16.00MiB	(used: 0.00B)

Data,single: Size:8.00MiB, Used:64.00KiB
   /dev/loop1	   8.00MiB

Metadata,single: Size:264.00MiB, Used:112.00KiB
   /dev/loop1	 264.00MiB

System,single: Size:4.00MiB, Used:16.00KiB
   /dev/loop1	   4.00MiB

Unallocated:
   /dev/loop1	   6.23GiB


# btrfs fi usage /mnt/b2
Overall:
    Device size:		   6.50GiB
    Device allocated:		  12.00MiB
    Device unallocated:		   6.49GiB
    Device missing:		     0.00B
    Used:			  32.00KiB
    Free (estimated):		   6.49GiB	(min: 6.49GiB)
    Data ratio:			      1.00
    Metadata ratio:		      1.00
    Global reserve:		   4.00MiB	(used: 0.00B)

Data+Metadata,single: Size:8.00MiB, Used:28.00KiB
   /dev/loop2	   8.00MiB

System,single: Size:4.00MiB, Used:4.00KiB
   /dev/loop2	   4.00MiB

Unallocated:
   /dev/loop2	   6.49GiB
Comment 1 Chris Murphy 2016-03-10 08:45:38 UTC
Interestingly, once written to, it seems to become normal. Identical data written to the two volumes, and now:

Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/loop1      6817792  4102028   2505092  63% /mnt/b1
/dev/loop2      6817792  4167048   2727548  61% /mnt/b2
Comment 2 Luis de Bethencourt 2016-03-23 17:28:06 UTC
This looks interesting, and a safe place for a new comer to experiment without breaking the file system.

I am planning to look into it. No promises. Will report soon on my progress.
Comment 3 Luis de Bethencourt 2016-03-24 11:38:09 UTC
I just reproduced this bug with Linux 4.4.6 (latest stable release). It isn't a regression introduced lately in the development repo.

Going to try slightly older stable releases to see when the regression was introduced.
Comment 4 Luis de Bethencourt 2016-03-24 17:15:37 UTC
Tried the empty mixed block group "-M" filesystem in a few more kernel versions. Bug happens in 4.4.6 but not in 4.4. This gives me a hint on when the regression was introduced.

Now I can start looking into the code and why it is missbehaving.

Will report again after the Easter weekend.
Comment 5 David Sterba 2016-03-24 17:22:39 UTC
The offending patch is known and has been backported to stable kernels, it was introduced in 4.5 cycle.

ca8a51b3a979d57b082b14eda38602b7f52d81d1
btrfs: statfs: report zero available if metadata are exhausted
Comment 6 Luis de Bethencourt 2016-03-24 18:57:06 UTC
Ah! that is even more information to know what is going on in btrfs/super.c:btrfs_statfs()

Thanks David for that information :)

Getting closer to a fix. Will report back on Monday/Tuesday.

Have a good Easter.
Comment 7 Luis de Bethencourt 2016-03-30 14:28:55 UTC
Created attachment 211091 [details]
patch to fix the issue that made df return 0 available space

Not sure if I should submit patches here directly or to the linux-btrfs@vger.kernel.org mailing list. Please let me know if I should submit them there in the future.
Comment 8 Luis de Bethencourt 2016-03-30 14:29:44 UTC
Created attachment 211101 [details]
Patch for a related issue in the same code. Protect f_bfree from a potential overflow.
Comment 9 David Sterba 2016-03-30 17:53:41 UTC
A patch attached for preview is ok, but please submit it to the mailinglist (with changelog and signed-off-by).
Comment 10 Luis de Bethencourt 2016-03-30 20:55:26 UTC
Thanks David. Sent to the mailing list.

I looked at previous closed bugs in here (bugzilla) and it wasn't clear to me since some had patches attached. Now I know for the future.
Comment 11 Luis de Bethencourt 2016-04-04 09:54:53 UTC
Hi,

Just as a heads up (or a reminder if you saw it already). I have sent the patch to fix this issue [0] and a second one to fix some related code [1].

They are waiting for review whenever you have time. Thank you :)

[0] https://lkml.org/lkml/2016/3/30/768
[1] https://lkml.org/lkml/2016/3/30/809
Comment 12 Luis de Bethencourt 2016-04-04 10:31:15 UTC
Nevermind!

I just realized the two patches have been merged into David's for-next branch. \o/

https://git.kernel.org/cgit/linux/kernel/git/kdave/linux.git/commit/?h=for-next&id=0e0998461def618b80beff44186e04097231c993
https://git.kernel.org/cgit/linux/kernel/git/kdave/linux.git/commit/?h=for-next&id=08cb1c628e9fa631754ba4de12cf433b6f65d805

This bug can be closed now.

Thank you David
Comment 13 David Sterba 2016-04-05 09:41:09 UTC
I have tested the patches and scheduled for next dev cycle. Thanks and closing.