Bug 218845

Summary: Kernel panic in 6.9.0 after changes in kernel/power/swap.c
Product: Power Management Reporter: Petri Kaukasoina (petri.kaukasoina)
Component: Hibernation/SuspendAssignee: Rafael J. Wysocki (rjw)
Status: RESOLVED PATCH_ALREADY_AVAILABLE    
Severity: normal CC: brauner, malowmas24, petri.kaukasoina
Priority: P3    
Hardware: All   
OS: Linux   
Kernel Version: 6.9.0 Subsystem:
Regression: Yes Bisected commit-id: 4379f91172f39d999919c8e8b2b5e1d665d8972d
Attachments: swap: yield device immediately

Description Petri Kaukasoina 2024-05-15 18:14:10 UTC
6.9.0 crashes on boot while 6.8.0 is ok.

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,1)

bisect result:

4379f91172f39d999919c8e8b2b5e1d665d8972d is the first bad commit
commit 4379f91172f39d999919c8e8b2b5e1d665d8972d
Author: Christian Brauner <brauner@kernel.org>
Date:   Tue Jan 23 14:26:23 2024 +0100

    power: port block device access to file
    
    Link: https://lore.kernel.org/r/20240123-vfs-bdev-file-v2-6-adbd023e19cc@kernel.org
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Reviewed-by: Jan Kara <jack@suse.cz>
    Signed-off-by: Christian Brauner <brauner@kernel.org>

 kernel/power/swap.c | 28 ++++++++++++++--------------

6.9.0 with only this reverted did not compile. This has something to do with reading a hibernation image from disk. I use a swap file, not a partition. The system was not hibernated, though. After I removed resume= and resume_offset= from the kernel command line, even 6.9.0 boots without panic.
Comment 1 Artem S. Tashkinov 2024-05-16 02:29:59 UTC
Christian please take a look.
Comment 2 Artem S. Tashkinov 2024-05-17 06:45:47 UTC
Christian please take a look
Comment 3 Petri Kaukasoina 2024-05-17 08:08:42 UTC
I replied to the message in linux-kernel list with this additional comment:


Afterwards I noticed that after I managed to boot 6.9.0, it hibernates 
and even wakes up OK, so it can read the hibernation image. The panic is 
there only when the system is coming up after rebooted. Then I need to 
edit out resume= and resume_offset= from grub to not get the kernel 
panic. (grub with non-UEFI legacy boot.)
Comment 4 Petri Kaukasoina 2024-05-17 10:47:54 UTC
This is how it looks on the screen (typed from a photograph):


Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,1)
CPU: 3 PID: 1 Comm: swapper/0 Not tainted 6.9.0 #1
Hardware Name: Acer Aspire E1-772G/EA70_HW, BIOS V1.03 01/28/2014
Call Trace:
 <TASK>
 dump_stack_lvl+0x4a/0x70
 panic+0x10f/0x29f
 mount_root_generic+0x1ca/0x270
 ? __pfx_kernel_init+0x10/0x10
 prepare_namespace+0x1e8/0x230
 kernel_init+0x16/0x130
 ret_from_fork+0x2d/0x50
 ? __pfx_kernel_init+0x10/0x10
 ret_from_fork_asm+0x1a/0x30
 </TASK>
Kernel Offset: 0x37000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,1) ]--
Comment 5 Christian Brauner 2024-05-21 14:36:44 UTC
(In reply to Artem S. Tashkinov from comment #2)
> Christian please take a look

Just got back from LSFMM on the weekend. I'll try and look tomorrow!
Comment 6 Petri Kaukasoina 2024-05-21 15:59:02 UTC
I changed the vga= mode in grub which made chars smaller and output slower, so I managed to see other messages before the kernel panic. There is error -16. Is it EBUSY?


VFS: Cannot open root device "/dev/sda1" or unknown-block(8,1): error -16
Please append a correct "root=" boot option: here are the available partitions:
<then lots of ram devices and sr0>
 driver: sd
  0801         976761543 sda1 7fcf396a-fa70-...
  0802              1007 sda2 05c5d621-...

List of all bdev filesystems:
 reiserfs
 ext3
 ext2
 ext4
...

<and then the kernel panic which was in Comment 4>
Comment 7 Petri Kaukasoina 2024-05-21 16:29:14 UTC
Still another hint... That EBUSY of the root fs gave me an idea.

I have all the needed drivers built-in in vmlinuz, so I do not use initrd at all. I thought that maybe with an initrd the kernel would not try to mount the "busy" HDD but the initramfs, instead. So I made an "empty" initrd without any modules, and it did boot without panic.

I guess my use is not too usual: a swap file for hibernation and no initrd. But it has worked before.
Comment 8 Christian Brauner 2024-05-21 19:03:10 UTC
Created attachment 306317 [details]
swap: yield device immediately
Comment 9 Christian Brauner 2024-05-21 19:03:31 UTC
(In reply to Christian Brauner from comment #8)
> Created attachment 306317 [details]
> swap: yield device immediately

Can you try the following patch, please?
Comment 10 Petri Kaukasoina 2024-05-21 20:36:32 UTC
Yes, your patch fixed the problem! Thanks.
Comment 11 Petri Kaukasoina 2024-05-24 08:54:43 UTC
Will this fix be in Linus's tree and stable linux-6.9.y?
Comment 12 Christian Brauner 2024-05-24 11:31:40 UTC
On Fri, May 24, 2024 at 08:54:43AM +0000, bugzilla-daemon@kernel.org wrote:
> https://bugzilla.kernel.org/show_bug.cgi?id=218845
> 
> --- Comment #11 from Petri Kaukasoina (petri.kaukasoina@tuni.fi) ---
> Will this fix be in Linus's tree and stable linux-6.9.y?

Yes, I'll send a pull request early next week.
Comment 13 Artem S. Tashkinov 2024-05-26 20:34:07 UTC
Thanks a ton for a quick fix, Christian.
Comment 14 Christian Brauner 2024-05-27 14:45:25 UTC
(In reply to Artem S. Tashkinov from comment #13)
> Thanks a ton for a quick fix, Christian.

You're welcome! Thanks for bringing this to my attention! I've sent a pull request including the fix earlier today!
Comment 15 Leslie Faulkner 2024-10-30 02:43:11 UTC
This kernel panic in 6.9.0 is due to commit 4379f91172f39d999919c8e8b2b5e1d665d8972d/ and https://lore.kernel.org/r/20240123-vfs-bdev-file-v2-6-adbd023e19cc@kernel.org/ https://amongusonline.co, which changed block device access for swap files. Reverting the commit failed to compile, but removing resume= and resume_offset= from the kernel command line allows 6.9.0 to boot without issues.