Bug 203693

Summary: btrfs-convert fails with ctree.c:2245: split_leaf on 4.6T, 2.5T used
Product: File System Reporter: Doncho Gunchev (dgunchev)
Component: btrfsAssignee: BTRFS virtual assignee (fs_btrfs)
Status: REOPENED ---    
Severity: normal CC: bugzilla.kernel.org, dgunchev, dsterba, mchsmith2018
Priority: P1    
Hardware: x86-64   
OS: Linux   
See Also: https://bugzilla.redhat.com/show_bug.cgi?id=1713035
Kernel Version: 5.0.16-300.fc30.x86_64 Subsystem:
Regression: No Bisected commit-id:
Attachments: fsck.ext4 -f -v and tune2fs -l

Description Doncho Gunchev 2019-05-23 16:11:33 UTC 
Created attachment 282925 [details]
fsck.ext4 -f -v and tune2fs -l

I tried converting ext4 file system with:

btrfs-convert -p -O extref,skinny-metadata -L /dev/mapper/luks-md127

The device is luks on top of raid1. I started with btrfs-progs-4.20.2-1.fc30.x86_64 (which has no patches) and later tried with btrfs-progs-5.1.

I think there was enough free space: Size=4620G, Used=3100G, Avail=1520G, Use%=67.1.

The output of the command is:
--- cut ---
create btrfs filesystem:
        blocksize: 4096
        nodesize:  16384
        features:  extref, skinny-metadata (default)
creating ext2 image file
Unable to find block group for 0
Unable to find block group for 0
Unable to find block group for 0
ctree.c:2245: split_leaf: BUG_ON `1` triggered, value 1
btrfs-convert(+0x16309)[0x561beff32309]
btrfs-convert(btrfs_search_slot+0xe85)[0x561beff337e5]
btrfs-convert(btrfs_csum_file_block+0x43f)[0x561beff45cff]
btrfs-convert(+0xe91b)[0x561beff2a91b]
btrfs-convert(main+0x1864)[0x561beff29b34]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f7b86a86f33]
btrfs-convert(_start+0x2e)[0x561beff2a4be]
Aborted (core dumped)
--- cut ---

before (btrfs-progs-4.20.2) it was failing at "copy_for_split at ctree.c:2127".

I also tried 'btrfs-convert -n -d -p -O extref,skinny-metadata ...', but it also failed.

I originally filled the bug in RH's bugzilla - https://bugzilla.redhat.com/show_bug.cgi?id=1713035 and got suggestion to report here. There a

I have a backup of all the data and will keep the setup for some time, can compile new versions and try debugging, but I have never looked into the kernel or btrfs.

Attaching fsck.ext4 and tune2fs -l...
Comment 1 Doncho Gunchev 2020-11-23 21:14:43 UTC 
I can't keep the setup any longer... and it is quite old most likely useless bug report.
Comment 2 mchsm 2023-02-03 06:51:26 UTC 
It should not be deemed as old useless bug. 

It is truly bug and will be beneficial for everyone if someone can seriously look into it 

# btrfs-convert /dev/mapper/vg11_media_new-lv11_media_new
create btrfs filesystem:
        blocksize: 4096
        nodesize:  16384
        features:  extref, skinny-metadata (default)
creating ext2 image file
Unable to find block group for 0
Unable to find block group for 0
Unable to find block group for 0
ctree.c:2245: split_leaf: BUG_ON `1` triggered, value 1
btrfs-convert(+0x18d22)[0x5638ad20bd22]
btrfs-convert(btrfs_search_slot+0x2a5)[0x5638ad20c635]
btrfs-convert(btrfs_csum_file_block+0x48f)[0x5638ad21ebff]
btrfs-convert(+0xf9f4)[0x5638ad2029f4]
btrfs-convert(main+0x1a3a)[0x5638ad201e5a]
/lib64/libc.so.6(__libc_start_main+0xef)[0x7f9125e9e2bd]
btrfs-convert(_start+0x2a)[0x5638ad20258a]
Aborted (core dumped)

# cat /etc/os-release
NAME="SLES"
VERSION="15-SP3"
VERSION_ID="15.3"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp3"
VARIANT_ID="sles-sap"

# uname -r
5.3.18-150300.59.76-default

# uname -v
#1 SMP Thu Jun 16 04:23:47 UTC 2022 (2cc2ade)

# nproc
4

# free -g
              total        used        free      shared  buff/cache   available
Mem:             30           0          28           0           1          29
Swap:             1           0           1

# lsblk -O /dev/mapper/vg11_media_new-lv11_media_new
NAME KNAME PATH MAJ:MIN FSAVAIL FSSIZE FSTYPE FSUSED FSUSE% FSVER MOUNTPOINT LABEL UUID                                 PTUUID PTTYPE PARTTYPE PARTTYPENAME PARTLABEL PARTUUID PARTFLAGS  RA RO RM HOTPLUG MODEL SERIAL   SIZE STATE OWNER GROUP MODE       ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED RQ-SIZE TYPE DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO WSAME WWN RAND PKNAME HCTL TRAN SUBSYSTEMS REV VENDOR ZONED DAX
vg11_media_new-lv11_media_new
     dm-4  /dev/mapper/vg11_media_new-lv11_media_new
                254:4                  ext4                 1.0                    210c476e-4a56-4685-992c-1001a2f2f2c3                                                                  512  0  0       0              124.8G runni root  disk  brw-rw----         0   4096   4096     512     512    0           128 lvm         0        0B       0B         0    0B        0                  block                 none    0
#
Comment 3 mchsm 2023-02-03 07:03:14 UTC 
# mount /dev/mapper/vg11_media_new-lv11_media_new /MEDIA_NEW/

# df -Th /dev/mapper/vg11_media_new-lv11_media_new
Filesystem                                Type  Size  Used Avail Use% Mounted on
/dev/mapper/vg11_media_new-lv11_media_new ext4  123G   74G   43G  64% /MEDIA_NEW

# lsblk /dev/nvme5n1
NAME                              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
nvme5n1                           259:9    0   125G  0 disk
└─nvme5n1p1                       259:12   0   125G  0 part
  └─vg11_media_new-lv11_media_new 254:4    0 124.8G  0 lvm

# uptime
 14:58:22  up 6 days 22:00,  3 users,  load average: 0.04, 0.03, 0.18

# tune2fs -l /dev/mapper/vg11_media_new-lv11_media_new
tune2fs 1.43.8 (1-Jan-2018)
Filesystem volume name:   <none>
Last mounted on:          /MEDIA_NEW
Filesystem UUID:          210c476e-4a56-4685-992c-1001a2f2f2c3
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              8183808
Block count:              32715776
Reserved block count:     1635788
Free blocks:              12849444
Free inodes:              8166263
First block:              0
Block size:               4096
Fragment size:            4096
Group descriptor size:    64
Reserved GDT blocks:      1024
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Flex block group size:    16
Filesystem created:       Tue Jan 10 19:10:43 2023
Last mount time:          Fri Feb  3 14:57:28 2023
Last write time:          Fri Feb  3 14:59:16 2023
Mount count:              5
Maximum mount count:      -1
Last checked:             Tue Jan 10 19:10:43 2023
Check interval:           0 (<none>)
Lifetime writes:          76 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     32
Desired extra isize:      32
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      eb01c2f1-a6ad-4334-914a-f38440a9db6e
Journal backup:           inode blocks
Comment 4 mchsm 2023-02-03 07:07:23 UTC 
# rpm -qa|grep btrfs
libbtrfs0-4.19.1-150300.18.5.1.x86_64
btrfsprogs-4.19.1-150300.18.5.1.x86_64
btrfsprogs-udev-rules-4.19.1-150300.18.5.1.noarch
btrfsmaintenance-0.4.2-3.3.1.noarch

# rpm -qi libbtrfs0
Name        : libbtrfs0
Version     : 4.19.1
Release     : 150300.18.5.1
Architecture: x86_64
Install Date: Fri Feb  3 14:09:28 2023
Group       : System/Libraries
Size        : 52160
License     : GPL-2.0
Signature   : RSA/SHA256, Thu Jul 14 12:04:17 2022, Key ID 70af9e8139db7c82
Source RPM  : btrfsprogs-4.19.1-150300.18.5.1.src.rpm
Build Date  : Thu Jul 14 12:02:39 2022
Build Host  : sheep64
Relocations : (not relocatable)
Packager    : https://www.suse.com/
Vendor      : SUSE LLC <https://www.suse.com/>
URL         : http://btrfs.wiki.kernel.org/index.php/Main_Page
Summary     : Library for interacting with Btrfs
Description :
This package contains the libbtrfs.so shared library needed for some
applications to interface with btrfs.
Distribution: SUSE Linux Enterprise 15
SGA2S4GEDD001-temp:~ # rpm -qi btrfsprogs
Name        : btrfsprogs
Version     : 4.19.1
Release     : 150300.18.5.1
Architecture: x86_64
Install Date: Fri Feb  3 14:09:29 2023
Group       : System/Filesystems
Size        : 3418254
License     : GPL-2.0
Signature   : RSA/SHA256, Thu Jul 14 12:04:17 2022, Key ID 70af9e8139db7c82
Source RPM  : btrfsprogs-4.19.1-150300.18.5.1.src.rpm
Build Date  : Thu Jul 14 12:02:39 2022
Build Host  : sheep64
Relocations : (not relocatable)
Packager    : https://www.suse.com/
Vendor      : SUSE LLC <https://www.suse.com/>
URL         : http://btrfs.wiki.kernel.org/index.php/Main_Page
Summary     : Utilities for the Btrfs filesystem
Description :
Utilities needed to create and maintain btrfs file systems under Linux.
Distribution: SUSE Linux Enterprise 15

# rpm -qi btrfsprogs-udev-rules
Name        : btrfsprogs-udev-rules
Version     : 4.19.1
Release     : 150300.18.5.1
Architecture: noarch
Install Date: Fri Feb  3 14:09:28 2023
Group       : System/Kernel
Size        : 387
License     : GPL-2.0
Signature   : RSA/SHA256, Thu Jul 14 12:04:17 2022, Key ID 70af9e8139db7c82
Source RPM  : btrfsprogs-4.19.1-150300.18.5.1.src.rpm
Build Date  : Thu Jul 14 12:02:39 2022
Build Host  : sheep64
Relocations : (not relocatable)
Packager    : https://www.suse.com/
Vendor      : SUSE LLC <https://www.suse.com/>
URL         : http://btrfs.wiki.kernel.org/index.php/Main_Page
Summary     : Udev rules for configuring btrfs file systems
Description :
This package contains the udev rule file for configuring device mapper
devices that are components of btrfs file systems.  It is meant to be
used with versions of udev that contain the "built-in" btrfs command
(v190 and newer).  Older versions of udev will call the version of
"btrfs ready" contained in the btrfsprogs package, which does the right
thing.
Distribution: SUSE Linux Enterprise 15

# rpm -qi btrfsmaintenance
Name        : btrfsmaintenance
Version     : 0.4.2
Release     : 3.3.1
Architecture: noarch
Install Date: Fri Feb  3 14:09:29 2023
Group       : System/Base
Size        : 54817
License     : GPL-2.0-only
Signature   : RSA/SHA256, Fri Jun  4 15:00:42 2021, Key ID 70af9e8139db7c82
Source RPM  : btrfsmaintenance-0.4.2-3.3.1.src.rpm
Build Date  : Fri Jun  4 15:00:14 2021
Build Host  : sheep05
Relocations : (not relocatable)
Packager    : https://www.suse.com/
Vendor      : SUSE LLC <https://www.suse.com/>
URL         : https://github.com/kdave/btrfsmaintenance
Summary     : Scripts for btrfs periodic maintenance tasks
Description :
Scripts for btrfs maintenance tasks like periodic scrub, balance, trim or defrag
on selected mountpoints or directories. Hints for periodic snapshot tuning (eg.
for snapper).
Distribution: SUSE Linux Enterprise 15
Comment 5 bugzilla.kernel.org 2023-08-23 13:51:25 UTC 
I get this with `btrfs-progs v6.2` (.deb from the standard repo, btrfs-progs 6.2-1 amd64) on Ubuntu 23.04, their latest kernel, 6.2.0-27-generic

Signature is slightly different, as it's in ctree.c:2455 but it's still 'split_leaf : BUG_ON `1` triggered, value 1`

It's an ext4 filesystem, `/dev/sda6        93G   74G   19G  80% /home`

tune2fs 1.47.0 (5-Feb-2023)
Filesystem volume name:   /home
Last mounted on:          /home
Filesystem UUID:          16608362-d206-443a-b433-5772d5ce7501
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              6201344
Block count:              24803072
Reserved block count:     248030
Overhead clusters:        437241
Free blocks:              5112709
Free inodes:              5580493
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1018
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
RAID stride:              32728
Flex block group size:    16
Filesystem created:       Thu Apr 18 18:57:51 2013
Last mount time:          Wed Aug 23 14:44:42 2023
Last write time:          Wed Aug 23 14:44:42 2023
Mount count:              881
Maximum mount count:      -1
Last checked:             Sun Apr 19 18:38:40 2020
Check interval:           0 (<none>)
Lifetime writes:          12 TB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
First orphan inode:       1849426
Default directory hash:   half_md4
Directory Hash Seed:      83c25cbf-18f9-47a1-8ee4-a6ce4151c65e
Journal backup:           inode blocks
Comment 6 bugzilla.kernel.org 2023-08-23 13:57:11 UTC 
I think this might be this bug : https://github.com/kdave/btrfs-progs/issues/312 which is something to do with even though the filesystem has 20% free space, btrfs can't actually use any of it