Bug 202803 - Hibernation does not work with BTRFS and swapfile on 5.0 kernel. Cannot find swap device.
Summary: Hibernation does not work with BTRFS and swapfile on 5.0 kernel. Cannot find ...
Status: RESOLVED CODE_FIX
Alias: None
Product: File System
Classification: Unclassified
Component: btrfs (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: BTRFS virtual assignee
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-07 06:39 UTC by Liberty
Modified: 2023-03-26 03:14 UTC (History)
10 users (show)

See Also:
Kernel Version: 5.0
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Liberty 2019-03-07 06:39:45 UTC
After linux 5.0, swapfile is supported on BTRFS. However I found the hibernation does not work correctly on 5.0 kernel.

Distro: Arch Linux
Setup: 
$ truncate -s 0 /swapfile
$ chattr +C /swapfile
$ fallocate -l 32G /swapfile
$ chmod 0600 /swapfile
$ mkswap /swapfile

# Add /swapfile to /etc/fstab
After rebooting
swapfile works correctly as swap memory

Run
$ filefrag -v /swapfile
Get the physical offset of first extent as 57869156

Added kernel parameter
resume=/dev/nvme1n1p2 resume_offset=57869156

After rebooting try hibernate with systemctl hibernate
The hibernation failed after blanking the screen and dmesg shows:
PM: Cannot find swap device, try swapon -a

BTW, the similar setup swapfile works correctly to hibernate with ext4 file system or swap partition.
Comment 1 Liberty 2019-03-07 06:42:45 UTC
BTW after hibernation failure and dmesg showing Cannot find swap device, try swapon -a. swapon shows the swapfile is functioning correctly. Running swapon -a does not help the issue.
Comment 2 David Sterba 2019-03-07 14:26:13 UTC
Adding Omar to CC.
Comment 3 Omar Sandoval 2019-03-07 19:17:12 UTC
This is because on Btrfs, the "physical" offset you get from filefrag isn't the real physical offset on disk; we have a virtual disk address space in order to support multiple devices. I'll hack up a tool you can use to get the actual physical offset on Btrfs.
Comment 4 Liberty 2019-03-07 21:52:29 UTC
Thanks Omar for the explanation. Let me know when you have the tool to get the actual physical offset on Btrfs.

Later, do we need to implement the functionality in e2fsprogs we can have the tool to get actual physical address?
Also we may need to update the man 5 btrfs.
Comment 5 Omar Sandoval 2019-03-08 07:15:01 UTC
Please try out https://github.com/osandov/osandov-linux/blob/master/scripts/btrfs_map_physical.c

It should compile with `gcc -O2 -o btrfs_map_physical btrfs_map_physical.c`. Run it with `./btrfs_map_physical /swapfile | column -ts $'\t'`. resume_offset should be the first physical offset.
Comment 6 Liberty 2019-03-08 07:37:12 UTC
Thanks Omar for the tool

I tried to run:

$ sudo ./btrfs_map_physical /swapfile | column -ts $'\t'                                                         

'FILE OFFSET  EXTENT TYPE  LOGICAL SIZE  LOGICAL OFFSET  PHYSICAL SIZE  DEVID  PHYSICAL OFFSET
0            regular      4096          248324816896    268435456      1      216112562176
4096         prealloc     268431360     248324820992    268431360      1      216112566272
268435456    prealloc     268435456     248593252352    268435456      1      216380997632
536870912    prealloc     268435456     248861687808    268435456      1      216649433088
805306368    prealloc     268435456     249130123264    268435456      1      241613930496
1073741824   prealloc     268435456     249398558720    268435456      1      241882365952
1342177280   prealloc     268435456     249666994176    268435456      1      242150801408
1610612736   prealloc     268435456     249935429632    268435456      1      242419236864
1879048192   prealloc     268435456     250203865088    268435456      1      242687672320
2147483648   prealloc     268435456     250472300544    268435456      1      242956107776
2415919104   prealloc     268435456     250740736000    268435456      1      243224543232
...

I used offset 216112562176 in kernel parameter:
resume=/dev/nvme1n1p2 resume_offset=216112562176

After rebooting.

$ swapon                                                                                                              
NAME           TYPE SIZE USED PRIO
/swapfile      file  16G   0B   -2


Run hibernate, I still got error:
[   26.828436] PM: Cannot find swap device, try swapon -a
[   26.828441] PM: Cannot get swap writer
Comment 7 Omar Sandoval 2019-03-08 08:56:34 UTC
Ah, apparently the unit is pages, so if this is x86, it should be 216112562176 / 4096 = 52761856. Also, `systemctl hibernate` seems to override those settings incorrectly. Can you try `echo disk > /sys/power/state` instead?
Comment 8 Omar Sandoval 2019-03-08 09:04:37 UTC
Yup, systemd overrides what you configure on the command line with the result of fiemap and stat, neither of which correspond to the physical device on Btrfs.

https://github.com/systemd/systemd/blob/c5e6f734c6477d1347a42074b208b4ef71c3095a/src/sleep/sleep.c#L80
Comment 9 candlelightdinner 2019-03-08 11:30:20 UTC
Thanks for the tool. Works for me with `echo disk > /sys/power/state` and


    $ sudo ./btrfs_map_physical /.swap/swapfile | column -ts $'\t' | head -n2
    FILE OFFSET  EXTENT TYPE  LOGICAL SIZE  LOGICAL OFFSET  PHYSICAL SIZE  DEVID  PHYSICAL OFFSET
    0            regular      4096          2407055360      268435456      1      2407055360


    GRUB_CMDLINE_LINUX="cryptdevice=PARTLABEL=cryptsystem:cryptroot resume=/dev/mapper/cryptroot resume_offset=587660"
Comment 10 Liberty 2019-03-08 21:50:16 UTC
Ah, I see. `echo disk > /sys/power/state` works with offset pages.

I filed a bug in systemd tracker also:

https://github.com/systemd/systemd/issues/11939
Comment 11 Zach Smith 2019-06-09 15:31:44 UTC
(In reply to Liberty from comment #10)
> Ah, I see. `echo disk > /sys/power/state` works with offset pages.
> 
> I filed a bug in systemd tracker also:
> 
> https://github.com/systemd/systemd/issues/11939

I've pushed a proposed fix for the systemd bug which checks for and honours previously set resume_offset values before attempting to calculate it. If anybody wants to give it a try to confirm it works (or not) with their setup, that would be helpful. https://github.com/systemd/systemd/pull/12760
Comment 12 Zach Smith 2019-06-30 19:35:34 UTC
(In reply to Zach Smith from comment #11)
> (In reply to Liberty from comment #10)
> > Ah, I see. `echo disk > /sys/power/state` works with offset pages.
> > 
> > I filed a bug in systemd tracker also:
> > 
> > https://github.com/systemd/systemd/issues/11939
> 
> I've pushed a proposed fix for the systemd bug which checks for and honours
> previously set resume_offset values before attempting to calculate it. If
> anybody wants to give it a try to confirm it works (or not) with their
> setup, that would be helpful. https://github.com/systemd/systemd/pull/12760

This issue should be resolved with the merge of PR https://github.com/systemd/systemd/pull/12760 and issue https://github.com/systemd/systemd/issues/11939 has been closed.
Comment 13 Liberty 2019-11-11 04:12:39 UTC
Yes, I have verified that systemd and btrfs swapfile working. The only pending issue is that we need to use Omar's script to calculate the physical offset of btrfs swapfile.
Comment 14 David Sterba 2023-01-20 14:15:49 UTC
The swapfile offset is available since btrfs-progs 6.1 as command 'inspect map-swapfile'.
Comment 15 Christoph Anton Mitterer 2023-03-26 03:14:46 UTC
@

Note You need to log in before you can comment on or make changes to this bug.