Bug 98821 - btrfs hung tasks on resume from hibernation
Summary: btrfs hung tasks on resume from hibernation
Status: RESOLVED UNREPRODUCIBLE
Alias: None
Product: File System
Classification: Unclassified
Component: btrfs (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Josef Bacik
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-05-24 09:40 UTC by Martin Steigerwald
Modified: 2016-03-20 11:08 UTC (History)
1 user (show)

See Also:
Kernel Version: 4.1-rc4
Tree: Mainline
Regression: No

Attachments
backtrace with hang on resume (1.14 MB, image/jpeg)
2015-05-24 09:40 UTC, Martin Steigerwald 		Details
Add an attachment (proposed patch, testcase, etc.)

Description Martin Steigerwald 2015-05-24 09:40:24 UTC 
Created attachment 177731 [details]
backtrace with hang on resume

With 4.1-rc4 additionally to kernel bug #94241 and fdo bug 86565 which appear to be gfx driver related¹², I now also that sometimes it just hangs on resume for an even longer time. This didn´t happen with 4.0 kernels.

This morning I didn´t look after the machine on switching on for a while and got a hint of what might be happening in these new cases. The kernel printed BTRFS hung task backtraces. I only have this screenshot as there is nothing in the logs.

It is 4.1-rc4 + a BTRFS trimming patch from Lutz Euler, as otherwise it wouldn´t trim at all on one of my filesystems on a ThinkPad T520 with / and /home on BTRFS RAID 1 on Dual SSD (SATA +  mSATA), and /daten just on mSATA SSD.

I bet this may be related to 

[Bug 90401] New: btrfs kworker thread uses up 100% of a Sandybridge core for minutes on random write into big file

as it seems that / filesystem has all device space allocated into chunks. Thus I will balance it, to free some device space.


merkaba:~> btrfs fi sh
Label: 'debian'  uuid: […]
        Total devices 2 FS bytes used 25.21GiB
        devid    1 size 30.00GiB used 30.00GiB path /dev/mapper/sata-debian
        devid    2 size 30.00GiB used 30.00GiB path /dev/mapper/msata-debian

Label: 'daten'  uuid: […]
        Total devices 1 FS bytes used 214.21GiB
        devid    1 size 230.00GiB used 221.02GiB path /dev/mapper/msata-daten

Label: 'home'  uuid: […]
        Total devices 2 FS bytes used 131.35GiB
        devid    1 size 170.00GiB used 149.06GiB path /dev/mapper/msata-home
        devid    2 size 170.00GiB used 149.06GiB path /dev/mapper/sata-home

btrfs-progs v4.0



merkaba:~> btrfs fi df /
Data, RAID1: total=27.99GiB, used=24.60GiB
System, RAID1: total=8.00MiB, used=16.00KiB
Metadata, RAID1: total=2.00GiB, used=625.12MiB
GlobalReserve, single: total=224.00MiB, used=0.00B
merkaba:~> btrfs fi df /home
Data, RAID1: total=144.03GiB, used=128.53GiB
System, RAID1: total=32.00MiB, used=48.00KiB
Metadata, RAID1: total=5.00GiB, used=2.82GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
merkaba:~> btrfs fi df /daten
Data, single: total=220.01GiB, used=213.80GiB
System, single: total=4.00MiB, used=48.00KiB
Metadata, single: total=1.01GiB, used=425.92MiB
GlobalReserve, single: total=144.00MiB, used=0.00B



merkaba:~> btrfs fi usage /
Overall:
    Device size:                  60.00GiB
    Device allocated:             60.00GiB
    Device unallocated:            2.00MiB
    Device missing:                  0.00B
    Used:                         50.41GiB
    Free (estimated):              3.39GiB      (min: 3.39GiB)
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              224.00MiB      (used: 0.00B)

Data,RAID1: Size:27.99GiB, Used:24.60GiB
   /dev/dm-2      27.99GiB
   /dev/mapper/sata-debian        27.99GiB

Metadata,RAID1: Size:2.00GiB, Used:625.12MiB
   /dev/dm-2       2.00GiB
   /dev/mapper/sata-debian         2.00GiB

System,RAID1: Size:8.00MiB, Used:16.00KiB
   /dev/dm-2       8.00MiB
   /dev/mapper/sata-debian         8.00MiB

Unallocated:
   /dev/dm-2       1.00MiB
   /dev/mapper/sata-debian         1.00MiB



merkaba:~> btrfs fi usage /home
Overall:
    Device size:                 340.00GiB
    Device allocated:            298.12GiB
    Device unallocated:           41.88GiB
    Device missing:                  0.00B
    Used:                        262.70GiB
    Free (estimated):             36.44GiB      (min: 36.44GiB)
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)

Data,RAID1: Size:144.03GiB, Used:128.53GiB
   /dev/dm-0     144.03GiB
   /dev/mapper/sata-home         144.03GiB

Metadata,RAID1: Size:5.00GiB, Used:2.82GiB
   /dev/dm-0       5.00GiB
   /dev/mapper/sata-home           5.00GiB

System,RAID1: Size:32.00MiB, Used:48.00KiB
   /dev/dm-0      32.00MiB
   /dev/mapper/sata-home          32.00MiB

Unallocated:
   /dev/dm-0      20.94GiB
   /dev/mapper/sata-home          20.94GiB


merkaba:~> btrfs fi usage /daten
Overall:
    Device size:                 230.00GiB
    Device allocated:            221.02GiB
    Device unallocated:            8.98GiB
    Device missing:                  0.00B
    Used:                        214.21GiB
    Free (estimated):             15.19GiB      (min: 15.19GiB)
    Data ratio:                       1.00
    Metadata ratio:                   1.00
    Global reserve:              144.00MiB      (used: 0.00B)

Data,single: Size:220.01GiB, Used:213.80GiB
   /dev/mapper/msata-daten       220.01GiB

Metadata,single: Size:1.01GiB, Used:425.92MiB
   /dev/mapper/msata-daten         1.01GiB

System,single: Size:4.00MiB, Used:48.00KiB
   /dev/mapper/msata-daten         4.00MiB

Unallocated:
   /dev/mapper/msata-daten         8.98GiB





[1] [Bug 94241] New: Blanks screen on hibernation but does not switch off the machine
https://bugzilla.kernel.org/show_bug.cgi?id=94241

[2] [Bug 86565] New: black screen after resume from hibernation since linux kernel 3.18
https://bugs.freedesktop.org/show_bug.cgi?id=86565
Comment 1 Martin Steigerwald 2015-05-24 10:11:12 UTC 
Lets see whether it is stable again after these rounds of balancing:


merkaba:~> btrfs fi sh /       
Label: 'debian'  uuid: 
        Total devices 2 FS bytes used 25.21GiB
        devid    1 size 30.00GiB used 30.00GiB path /dev/mapper/sata-debian
        devid    2 size 30.00GiB used 30.00GiB path /dev/mapper/msata-debian

btrfs-progs v4.0
merkaba:~> btrfs fi df /
Data, RAID1: total=27.99GiB, used=24.60GiB
System, RAID1: total=8.00MiB, used=16.00KiB
Metadata, RAID1: total=2.00GiB, used=625.12MiB
GlobalReserve, single: total=224.00MiB, used=0.00B
merkaba:~> btrfs balance start -musage=80 /
ERROR: error during balancing '/' - No space left on device
There may be more info in syslog - try dmesg | tail


merkaba:~> btrfs fi sh / ; btrfs fi df /   
Label: 'debian'  uuid: 
        Total devices 2 FS bytes used 25.21GiB
        devid    1 size 30.00GiB used 28.99GiB path /dev/mapper/sata-debian
        devid    2 size 30.00GiB used 28.99GiB path /dev/mapper/msata-debian

btrfs-progs v4.0
Data, RAID1: total=27.97GiB, used=24.60GiB
System, RAID1: total=19.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=624.72MiB
GlobalReserve, single: total=224.00MiB, used=0.00B

merkaba:~> btrfs balance start -dusage=80 /
Done, had to relocate 1 out of 30 chunks


merkaba:~> btrfs fi sh / ; btrfs fi df /   
Label: 'debian'  uuid: 
        Total devices 2 FS bytes used 25.20GiB
        devid    1 size 30.00GiB used 28.99GiB path /dev/mapper/sata-debian
        devid    2 size 30.00GiB used 28.99GiB path /dev/mapper/msata-debian

btrfs-progs v4.0
Data, RAID1: total=27.97GiB, used=24.60GiB
System, RAID1: total=19.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=622.81MiB
GlobalReserve, single: total=208.00MiB, used=16.00KiB


merkaba:~> btrfs balance start -dusage=70 /
Done, had to relocate 1 out of 30 chunks

merkaba:~> btrfs fi sh / ; btrfs fi df /   
Label: 'debian'  uuid: 
        Total devices 2 FS bytes used 25.20GiB
        devid    1 size 30.00GiB used 28.99GiB path /dev/mapper/sata-debian
        devid    2 size 30.00GiB used 28.99GiB path /dev/mapper/msata-debian

btrfs-progs v4.0
Data, RAID1: total=27.97GiB, used=24.59GiB
System, RAID1: total=19.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=623.78MiB
GlobalReserve, single: total=208.00MiB, used=976.00KiB


merkaba:~> btrfs balance start -dusage=50 /
Done, had to relocate 1 out of 30 chunks

merkaba:~> btrfs fi sh / ; btrfs fi df /   
Label: 'debian'  uuid: 
        Total devices 2 FS bytes used 25.20GiB
        devid    1 size 30.00GiB used 28.99GiB path /dev/mapper/sata-debian
        devid    2 size 30.00GiB used 28.99GiB path /dev/mapper/msata-debian

btrfs-progs v4.0
Data, RAID1: total=27.97GiB, used=24.59GiB
System, RAID1: total=19.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=622.56MiB
GlobalReserve, single: total=208.00MiB, used=80.00KiB


merkaba:~> btrfs balance start -dusage=30 /
Done, had to relocate 0 out of 30 chunks
merkaba:~> btrfs balance start -dusage=20 /
Done, had to relocate 0 out of 30 chunks
merkaba:~> btrfs balance start -dusage=10 /
Done, had to relocate 0 out of 30 chunks
merkaba:~> btrfs balance start -dusage=90 /
Done, had to relocate 15 out of 30 chunks

merkaba:~> btrfs fi sh / ; btrfs fi df /   
Label: 'debian'  uuid: 
        Total devices 2 FS bytes used 25.20GiB
        devid    1 size 30.00GiB used 26.99GiB path /dev/mapper/sata-debian
        devid    2 size 30.00GiB used 26.99GiB path /dev/mapper/msata-debian

btrfs-progs v4.0
Data, RAID1: total=25.97GiB, used=24.60GiB
System, RAID1: total=19.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=621.27MiB
GlobalReserve, single: total=208.00MiB, used=1.03MiB
Comment 2 Martin Steigerwald 2016-03-20 11:08:20 UTC 
I didn´t see this in a long time anymore. I saw the more general hung task thing

[Bug 90401] New: btrfs kworker thread uses up 100% of a Sandybridge core for minutes on random write into big file

at least upto 4.3 AFAIR.

But I didn´t see this anymore. So closing.
Note You need to log in before you can comment on or make changes to this bug.