Bug 218471

Summary:	F2FS fails to mount rw at boot with "invalid zstd compress level: 6"
Product:	File System	Reporter:	Dhya Pacifica (dhya)
Component:	f2fs	Assignee:	Default virtual assignee for f2fs (filesystem_f2fs)
Status:	NEW ---
Severity:	high	CC:	carnil, chao, jaegeuk
Priority:	P3
Hardware:	All
OS:	Linux
Kernel Version:	6.1.76	Subsystem:
Regression:	Yes	Bisected commit-id:

Description Dhya Pacifica 2024-02-08 09:09:18 UTC

Using Debian 12.5 "Bookworm" and updated to most-recent stable release kernel "linux-image-6.1.0-18-amd64" which changelog says:
  linux-signed-amd64 (6.1.76+1) bookworm; urgency=medium
  * Sign kernel from linux 6.1.76-1

When it boots the f2fs filesystem fails to mount read-write and the boot journal shows:
  kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6

This is the fstab entry for the corresponding partition:
  UUID=XXXXXXXXXXX   /   f2fs    compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime   0       1

Under the previous kernel the partition correctly mounts and the 'mount' command shows:
  /dev/nvme0n1p6 on / type f2fs (rw,relatime,lazytime,background_gc=on,gc_merge,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=zstd:6,compress_log_size=2,compress_chksum,compress_mode=fs,atgc,discard_unit=block,memory=normal)

I was able to remount the filesystem read-write with:
  sudo mount -o remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,discard_unit=block,memory=normal /dev/nvme0n1p6 /

and from there was able to update /etc/default/grub to boot the previous kernel:
  GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-17-amd64"

I believe this bug may have been introduced with this patch: 
  https://www.spinics.net/lists/stable-commits/msg329957.html

Comment 1 Dhya Pacifica 2024-02-08 09:11:56 UTC

The changelog for the Debian linux-image-6.1.0-18-amd64 kernel package lists:

    - f2fs: clean up i_compress_flag and i_compress_level usage
    - f2fs: convert to use bitmap API
    - f2fs: assign default compression level
    - f2fs: set the default compress_level on ioctl

Comment 2 Salvatore Bonaccorso 2024-02-08 20:58:27 UTC

regressions list report: https://lore.kernel.org/regressions/ZcU3VCrt9VOpuFUq@eldamar.lan/T/#u

Debian downstream bugreport: https://bugs.debian.org/1063422

Comment 3 Jaegeuk Kim 2024-02-09 01:42:51 UTC

Thanks. Let me check.

Comment 4 Chao Yu 2024-02-13 13:52:56 UTC

(In reply to Dhya Pacifica from comment #0)
> Using Debian 12.5 "Bookworm" and updated to most-recent stable release
> kernel "linux-image-6.1.0-18-amd64" which changelog says:
>   linux-signed-amd64 (6.1.76+1) bookworm; urgency=medium
>   * Sign kernel from linux 6.1.76-1
> 
> When it boots the f2fs filesystem fails to mount read-write and the boot
> journal shows:
>   kernel: F2FS-fs (nvme0n1p6): invalid zstd compress level: 6
> 
> This is the fstab entry for the corresponding partition:
>   UUID=XXXXXXXXXXX   /   f2fs   
> compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime   0       1
> 
> Under the previous kernel the partition correctly mounts and the 'mount'
> command shows:
>   /dev/nvme0n1p6 on / type f2fs
> (rw,relatime,lazytime,background_gc=on,gc_merge,discard,no_heap,user_xattr,
> inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,
> mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,
> fsync_mode=posix,compress_algorithm=zstd:6,compress_log_size=2,
> compress_chksum,compress_mode=fs,atgc,discard_unit=block,memory=normal)
> 
> I was able to remount the filesystem read-write with:
>   sudo mount -o
> remount,rw,relatime,lazytime,background_gc=on,discard,no_heap,user_xattr,
> inline_xattr,acl,inline_data,inline_dentry,extent_cache,mode=adaptive,
> active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,
> compress_algorithm=lz4,compress_log_size=2,compress_mode=fs,atgc,
> discard_unit=block,memory=normal /dev/nvme0n1p6 /
> 
> and from there was able to update /etc/default/grub to boot the previous
> kernel:
>   GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with
> Linux 6.1.0-17-amd64"
> 
> I believe this bug may have been introduced with this patch: 
>   https://www.spinics.net/lists/stable-commits/msg329957.html

Can you please try below fixes:

[PATCH 6.1] f2fs: add helper to check compression level
https://lore.kernel.org/linux-f2fs-devel/20240212160530.1017205-1-chao@kernel.org/T/#u

[PATCH] f2fs: compress: fix to check zstd compress level correctly in mount option
https://lore.kernel.org/linux-f2fs-devel/20240212160818.1020903-1-chao@kernel.org/T/#u