Bug 202495

Summary: oops in fs/f2fs/checkpoint.c:150 f2fs_is_valid_blkaddr
Product: File System Reporter: JWalker (jw7779)
Component: f2fsAssignee: Default virtual assignee for f2fs (filesystem_f2fs)
Status: RESOLVED PATCH_ALREADY_AVAILABLE    
Severity: normal CC: chao, hacks+kernel, phiber2000
Priority: P1    
Hardware: ARM   
OS: Linux   
Kernel Version: 4.14.97 Subsystem:
Regression: No Bisected commit-id:
Attachments: attachment-6765-0.html

Description JWalker 2019-02-02 15:48:32 UTC 
DMESG buffer filled with several oops's on boot

[   24.810633] ------------[ cut here ]------------
[   24.810690] WARNING: CPU: 0 PID: 140 at fs/f2fs/checkpoint.c:150 f2fs_is_valid_blkaddr+0x1d8/0x25c
[   24.810698] Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables arc4 rt2800usb rt2800lib rt2x00usb rt2x00lib crc_ccitt mac80211 cfg80211 rfkill rtc_ds3232 hwmon i2c_dev i2c_bcm2835
[   24.810836] CPU: 0 PID: 140 Comm: ls Tainted: G        W       4.14.97+ #34
[   24.810842] Hardware name: BCM2835
[   24.810898] [<c0016218>] (unwind_backtrace) from [<c0013b48>] (show_stack+0x20/0x24)
[   24.810937] [<c0013b48>] (show_stack) from [<c057b694>] (dump_stack+0x20/0x28)
[   24.810967] [<c057b694>] (dump_stack) from [<c0021ef0>] (__warn+0xe4/0x10c)
[   24.810988] [<c0021ef0>] (__warn) from [<c0021fe4>] (warn_slowpath_null+0x30/0x38)
[   24.811010] [<c0021fe4>] (warn_slowpath_null) from [<c027c974>] (f2fs_is_valid_blkaddr+0x1d8/0x25c)
[   24.811043] [<c027c974>] (f2fs_is_valid_blkaddr) from [<c026af84>] (f2fs_iget+0x764/0x1110)
[   24.811064] [<c026af84>] (f2fs_iget) from [<c026e6fc>] (f2fs_lookup+0xd4/0x1cc)
[   24.811093] [<c026e6fc>] (f2fs_lookup) from [<c0168744>] (lookup_slow+0x94/0x13c)
[   24.811113] [<c0168744>] (lookup_slow) from [<c0168fa4>] (walk_component+0x1bc/0x2e8)
[   24.811134] [<c0168fa4>] (walk_component) from [<c016a9cc>] (path_lookupat+0x8c/0x1f0)
[   24.811158] [<c016a9cc>] (path_lookupat) from [<c016d474>] (filename_lookup+0xa0/0xfc)
[   24.811179] [<c016d474>] (filename_lookup) from [<c016d5bc>] (user_path_at_empty+0x54/0x5c)
[   24.811197] [<c016d5bc>] (user_path_at_empty) from [<c0162974>] (vfs_statx+0x68/0xbc)
[   24.811215] [<c0162974>] (vfs_statx) from [<c01631bc>] (SyS_lstat64+0x38/0x58)
[   24.811236] [<c01631bc>] (SyS_lstat64) from [<c000fba0>] (ret_fast_syscall+0x0/0x28)
[   24.811245] ---[ end trace c72216233c81afbe ]---
Comment 1 Chao Yu 2019-02-15 08:28:07 UTC 
Hi,

Do you pick up 4.14 kernel from AOSP, or just kernel.org? did you try fsck on this image?
Comment 2 JWalker 2019-02-15 18:54:04 UTC 
Hi Chao,

The kernal code was taken from the raspberry pi git repo, and then the
incremental patches applied.

Yes, fsck has been applied. Please see below the output of the both volumes.

Please let me know if you require any more info.

Regards

John

sudo fsck -t f2fs /dev/sdc3
fsck from util-linux 2.31.1
Info: [/dev/sdc3] Disk Model: USB3.0 CRW   -SD1.00
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 4194304 (2048 MB)
Info: MKFS version
  "Linux version 4.13.0-43-generic (buildd@lgw01-amd64-041) (gcc
version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9))
#48~16.04.1-Ubuntu SMP Thu May 17 13:00:11 UTC 2018"
Info: FSCK version
  from "Linux version 4.15.0-43-generic (buildd@lgw01-amd64-001) (gcc
version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #46-Ubuntu SMP Thu Dec 6
14:45:28 UTC 2018"
    to "Linux version 4.15.0-45-generic (buildd@lgw01-amd64-031) (gcc
version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #48-Ubuntu SMP Tue Jan 29
16:28:13 UTC 2019"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 4194304 (2048 MB)
Info: CKPT version = 15a2c
Info: checkpoint state = 41 :  crc unmount

[FSCK] Unreachable nat entries                        [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking                [Ok..]
[FSCK] Hard link checking for regular file            [Ok..] [0x1]
[FSCK] valid_block_count matching with CP             [Ok..] [0x87a0]
[FSCK] valid_node_count matcing with CP (de lookup)   [Ok..] [0x107b]
[FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x107b]
[FSCK] valid_inode_count matched with CP              [Ok..] [0x1073]
[FSCK] free segment_count matched with CP             [Ok..] [0x296]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Ok..]

Done.
sudo fsck -t f2fs /dev/sdc4
fsck from util-linux 2.31.1
Info: [/dev/sdc4] Disk Model: USB3.0 CRW   -SD1.00
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 3055616 (1492 MB)
Info: MKFS version
  "Linux version 4.13.0-43-generic (buildd@lgw01-amd64-041) (gcc
version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9))
#48~16.04.1-Ubuntu SMP Thu May 17 13:00:11 UTC 2018"
Info: FSCK version
  from "Linux version 4.15.0-43-generic (buildd@lgw01-amd64-001) (gcc
version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #46-Ubuntu SMP Thu Dec 6
14:45:28 UTC 2018"
    to "Linux version 4.15.0-45-generic (buildd@lgw01-amd64-031) (gcc
version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #48-Ubuntu SMP Tue Jan 29
16:28:13 UTC 2019"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 3055616 (1492 MB)
Info: CKPT version = b
Info: checkpoint state = 45 :  crc compacted_summary unmount

[FSCK] Unreachable nat entries                        [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking                [Ok..]
[FSCK] Hard link checking for regular file            [Ok..] [0x0]
[FSCK] valid_block_count matching with CP             [Ok..] [0x2]
[FSCK] valid_node_count matcing with CP (de lookup)   [Ok..] [0x1]
[FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x1]
[FSCK] valid_inode_count matched with CP              [Ok..] [0x1]
[FSCK] free segment_count matched with CP             [Ok..] [0x2d9]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Ok..]

Done.

On Fri, 15 Feb 2019 at 08:28, <bugzilla-daemon@bugzilla.kernel.org> wrote:
>
> https://bugzilla.kernel.org/show_bug.cgi?id=202495
>
> Chao Yu (chao@kernel.org) changed:
>
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |chao@kernel.org
>
> --- Comment #1 from Chao Yu (chao@kernel.org) ---
> Hi,
>
> Do you pick up 4.14 kernel from AOSP, or just kernel.org? did you try fsck on
> this image?
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 3 Chao Yu 2019-02-18 01:30:21 UTC 
(In reply to JWalker from comment #2)
> Hi Chao,
> 
> The kernal code was taken from the raspberry pi git repo, and then the
> incremental patches applied.

What patches do you applied? some fixing patches?

> 
> Yes, fsck has been applied. Please see below the output of the both volumes.
> 
> Please let me know if you require any more info.

...

> [FSCK] Unreachable nat entries                        [Ok..] [0x0]
> [FSCK] SIT valid block bitmap checking                [Ok..]
> [FSCK] Hard link checking for regular file            [Ok..] [0x1]
> [FSCK] valid_block_count matching with CP             [Ok..] [0x87a0]
> [FSCK] valid_node_count matcing with CP (de lookup)   [Ok..] [0x107b]
> [FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x107b]
> [FSCK] valid_inode_count matched with CP              [Ok..] [0x1073]
> [FSCK] free segment_count matched with CP             [Ok..] [0x296]
> [FSCK] next block offset is free                      [Ok..]
> [FSCK] fixing SIT types
> [FSCK] other corrupted bugs                           [Ok..]

I didn't see any corruption there, does this image still panic your kernel?
Comment 4 JWalker 2019-02-18 18:56:11 UTC 
Created attachment 281189 [details]
attachment-6765-0.html

Hi Chao,
I’ve applied all incremental patches up to, including 4.14.101. I can also
confirm that my build is a %100 match between the .xz file on kernel.org (
with respect to fs/* ).
Yes , totally repeatable. Actually it appears to panic about 8 times on
reboot.
Also the panic is not kernel version specific as I’ve tried multiple kernel
versions. Same result. I’ve also reformatted a new sdcard, tar’d up all the
data on the old card, and transferred to the new one, and all is fine.
One other separate issue, should the
/sys/fs/f2fs/mmcblkXpY/lifetime_write_kbytes work in 4.14? Mine doesn’t,
but I have seen it work in Ubuntu 18.4 with 4.15 kernel. On reboot values
return to 0.

Hope this helps
John

On Mon, 18 Feb 2019 at 01:30, <bugzilla-daemon@bugzilla.kernel.org> wrote:

> https://bugzilla.kernel.org/show_bug.cgi?id=202495
>
> --- Comment #3 from Chao Yu (chao@kernel.org) ---
> (In reply to JWalker from comment #2)
> > Hi Chao,
> >
> > The kernal code was taken from the raspberry pi git repo, and then the
> > incremental patches applied.
>
> What patches do you applied? some fixing patches?
>
> >
> > Yes, fsck has been applied. Please see below the output of the both
> volumes.
> >
> > Please let me know if you require any more info.
>
> ...
>
> > [FSCK] Unreachable nat entries                        [Ok..] [0x0]
> > [FSCK] SIT valid block bitmap checking                [Ok..]
> > [FSCK] Hard link checking for regular file            [Ok..] [0x1]
> > [FSCK] valid_block_count matching with CP             [Ok..] [0x87a0]
> > [FSCK] valid_node_count matcing with CP (de lookup)   [Ok..] [0x107b]
> > [FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x107b]
> > [FSCK] valid_inode_count matched with CP              [Ok..] [0x1073]
> > [FSCK] free segment_count matched with CP             [Ok..] [0x296]
> > [FSCK] next block offset is free                      [Ok..]
> > [FSCK] fixing SIT types
> > [FSCK] other corrupted bugs                           [Ok..]
>
> I didn't see any corruption there, does this image still panic your kernel?
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 5 phiber2000 2019-03-18 23:26:26 UTC 
The issue was observed in the kernel version 4.14.98 an is still present in 4.14.106.
We successfully backported a commit of Chao Yu from 4.19 to resolve the issue.
Maybe it should be considered to be done here in the kernel.org upstream, too.
Further information: https://github.com/raspberrypi/linux/issues/2894
Comment 6 Chao Yu 2019-07-17 02:08:58 UTC 
Was fixed with a backported patch ("f2fs: fix to skip verifying block address for non-regular inode")

Close it.
Comment 7 phiber2000 2019-07-17 20:56:45 UTC 
(In reply to Chao Yu from comment #6)
> Was fixed with a backported patch ("f2fs: fix to skip verifying block
> address for non-regular inode")

Maybe my mistake, but I can't find the backported patch in the v4.14.y branch.
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/?h=linux-4.14.y&qt=grep&q=f2fs

Could you just check that?