201525 – Broken btrfs, unable to repair or rebalance

Bug 201525 - Broken btrfs, unable to repair or rebalance

Summary: Broken btrfs, unable to repair or rebalance

Status:	NEW

Alias:	None

Product:	File System
Classification:	Unclassified
Component:	btrfs (show other bugs)
Hardware:	Intel Linux

Importance:	P1 normal
Assignee:	BTRFS virtual assignee

URL:
Keywords:

Depends on:
Blocks:

Reported:	2018-10-26 14:23 UTC by Seb Lu
Modified:	2018-11-18 19:33 UTC (History)
CC List:	0 users

See Also:
Kernel Version:	4.19.0
Tree:	Mainline
Regression:	No

Attachments
Add an attachment (proposed patch, testcase, etc.)

Description Seb Lu 2018-10-26 14:23:16 UTC

Hello,

I run a btrfs on 6 HDD of 6TB. Data/Medatadata was in RAID1 mode.
Frequently, the whole server freeze when the backuppc process do its homeworks.

A scrub was running fine without errors.
A btrfs check reveal errors.
A btrfs check --repair make the btrfs check process to segfault.

I tried to rebalance of metadata to RAID10 mode. Which finished successfully.
Then, I tried to reblance data to RAID10 mode. But the kernel crash before it finish.

I updated to 4.19.0 but still crash during the balance.

The dmesg when crashing display message like this:

Oct 26 16:05:18 black.seblu.net kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Oct 26 16:05:18 black.seblu.net kernel: rcu:         Tasks blocked on level-0 rcu_node (CPUs 0-3): P818
Oct 26 16:05:18 black.seblu.net kernel: rcu:         (detected by 0, t=60003 jiffies, g=629029, q=89120)
Oct 26 16:05:18 black.seblu.net kernel: BackupPC_backup D    0   818    754 0x80000000
Oct 26 16:05:18 black.seblu.net kernel: Call Trace:
Oct 26 16:05:18 black.seblu.net kernel:  ? __schedule+0x1bd/0x5d0
Oct 26 16:05:18 black.seblu.net kernel:  ? enqueue_task_fair+0xa6/0x5f0
Oct 26 16:05:18 black.seblu.net kernel:  schedule+0x2a/0x80
Oct 26 16:05:18 black.seblu.net kernel:  schedule_timeout+0x1b5/0x290
Oct 26 16:05:18 black.seblu.net kernel:  ? check_preempt_curr+0x75/0x90
Oct 26 16:05:18 black.seblu.net kernel:  ? ttwu_do_wakeup.isra.7+0xd/0x90
Oct 26 16:05:18 black.seblu.net kernel:  ? try_to_wake_up+0x26e/0x370
Oct 26 16:05:18 black.seblu.net kernel:  wait_for_common+0x15a/0x190
Oct 26 16:05:18 black.seblu.net kernel:  ? wake_up_q+0x70/0x70
Oct 26 16:05:18 black.seblu.net kernel:  virt_efi_set_variable+0xe9/0x130
Oct 26 16:05:18 black.seblu.net kernel:  ? efi_call_virt_check_flags+0x60/0x60
Oct 26 16:05:18 black.seblu.net kernel:  efi_query_variable_store+0xeb/0x1b0
Oct 26 16:05:18 black.seblu.net kernel:  efivar_entry_set_safe+0xac/0x1d0
Oct 26 16:05:18 black.seblu.net kernel:  efi_pstore_write+0xf7/0x140
Oct 26 16:05:18 black.seblu.net kernel:  pstore_dump+0x132/0x340
Oct 26 16:05:18 black.seblu.net kernel:  ? irq_work_queue+0xa1/0xc0
Oct 26 16:05:18 black.seblu.net kernel:  kmsg_dump+0xb7/0xe0
Oct 26 16:05:18 black.seblu.net kernel:  oops_end+0x5f/0xc0
Oct 26 16:05:18 black.seblu.net kernel:  do_trap+0x6f/0xe0
Oct 26 16:05:18 black.seblu.net kernel:  do_error_trap+0xaf/0x130
Oct 26 16:05:18 black.seblu.net kernel:  ? btrfs_delete_delayed_dir_index+0x28f/0x2a0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  invalid_op+0x14/0x20
Oct 26 16:05:18 black.seblu.net kernel: RIP: 0010:btrfs_delete_delayed_dir_index+0x28f/0x2a0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel: Code: 8b 43 10 41 b9 ef ff ff ff 4c 89 ea 48 c7 c6 20 1d 45 c0 4c 8b 03 48 8b 88 38 03 00 00 48 8b 04 24 48 8b 78 50 e8 4e 30 02 00 <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 53 48 89 fb e8
Oct 26 16:05:18 black.seblu.net kernel: RSP: 0018:ffff9e5b41a03d40 EFLAGS: 00010286
Oct 26 16:05:18 black.seblu.net kernel: RAX: 0000000000000000 RBX: ffff9742109e2730 RCX: 0000000000000000
Oct 26 16:05:18 black.seblu.net kernel: RDX: 0000000000000000 RSI: ffff974217196388 RDI: ffff974217196388
Oct 26 16:05:18 black.seblu.net kernel: RBP: ffff9740058d1500 R08: 0000000000000000 R09: 000000000000035e
Oct 26 16:05:18 black.seblu.net kernel: R10: ffff9742051e0aa0 R11: 0000000000000001 R12: ffff9742109e2778
Oct 26 16:05:18 black.seblu.net kernel: R13: 000000000000002a R14: ffff9741fd2ff490 R15: ffff974112c43580
Oct 26 16:05:18 black.seblu.net kernel:  __btrfs_unlink_inode+0x280/0x3e0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  btrfs_unlink_inode+0x12/0x40 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  btrfs_unlink+0x87/0xd0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  vfs_unlink+0xea/0x190
Oct 26 16:05:18 black.seblu.net kernel:  do_unlinkat+0x220/0x300
Oct 26 16:05:18 black.seblu.net kernel:  do_syscall_64+0x49/0xf0
Oct 26 16:05:18 black.seblu.net kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Oct 26 16:05:18 black.seblu.net kernel: RIP: 0033:0x7f01a598b32b
Oct 26 16:05:18 black.seblu.net kernel: Code: Bad RIP value.
Oct 26 16:05:18 black.seblu.net kernel: RSP: 002b:00007fffd6de10e8 EFLAGS: 00000206 ORIG_RAX: 0000000000000057
Oct 26 16:05:18 black.seblu.net kernel: RAX: ffffffffffffffda RBX: 000055ef5447c260 RCX: 00007f01a598b32b
Oct 26 16:05:18 black.seblu.net kernel: RDX: 00007fffd6de1120 RSI: 00007fffd6de1120 RDI: 000055ef54b605c0
Oct 26 16:05:18 black.seblu.net kernel: RBP: 000055ef54bd3088 R08: 0000000000000001 R09: 0000000000000000
Oct 26 16:05:18 black.seblu.net kernel: R10: 000000000000015c R11: 0000000000000206 R12: 0000000000000001
Oct 26 16:05:18 black.seblu.net kernel: R13: 000055ef54bd3088 R14: 000055ef54b605c0 R15: 0000000000000000
Oct 26 16:05:18 black.seblu.net kernel: BackupPC_backup D    0   818    754 0x80000000
Oct 26 16:05:18 black.seblu.net kernel: Call Trace:
Oct 26 16:05:18 black.seblu.net kernel:  ? __schedule+0x1bd/0x5d0
Oct 26 16:05:18 black.seblu.net kernel:  ? enqueue_task_fair+0xa6/0x5f0
Oct 26 16:05:18 black.seblu.net kernel:  schedule+0x2a/0x80
Oct 26 16:05:18 black.seblu.net kernel:  schedule_timeout+0x1b5/0x290
Oct 26 16:05:18 black.seblu.net kernel:  ? check_preempt_curr+0x75/0x90
Oct 26 16:05:18 black.seblu.net kernel:  ? ttwu_do_wakeup.isra.7+0xd/0x90
Oct 26 16:05:18 black.seblu.net kernel:  ? try_to_wake_up+0x26e/0x370
Oct 26 16:05:18 black.seblu.net kernel:  wait_for_common+0x15a/0x190
Oct 26 16:05:18 black.seblu.net kernel:  ? wake_up_q+0x70/0x70
Oct 26 16:05:18 black.seblu.net kernel:  virt_efi_set_variable+0xe9/0x130
Oct 26 16:05:18 black.seblu.net kernel:  ? efi_call_virt_check_flags+0x60/0x60
Oct 26 16:05:18 black.seblu.net kernel:  efi_query_variable_store+0xeb/0x1b0
Oct 26 16:05:18 black.seblu.net kernel:  efivar_entry_set_safe+0xac/0x1d0
Oct 26 16:05:18 black.seblu.net kernel:  efi_pstore_write+0xf7/0x140
Oct 26 16:05:18 black.seblu.net kernel:  pstore_dump+0x132/0x340
Oct 26 16:05:18 black.seblu.net kernel:  ? irq_work_queue+0xa1/0xc0
Oct 26 16:05:18 black.seblu.net kernel:  kmsg_dump+0xb7/0xe0
Oct 26 16:05:18 black.seblu.net kernel:  oops_end+0x5f/0xc0
Oct 26 16:05:18 black.seblu.net kernel:  do_trap+0x6f/0xe0
Oct 26 16:05:18 black.seblu.net kernel:  do_error_trap+0xaf/0x130
Oct 26 16:05:18 black.seblu.net kernel:  ? btrfs_delete_delayed_dir_index+0x28f/0x2a0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  invalid_op+0x14/0x20
Oct 26 16:05:18 black.seblu.net kernel: RIP: 0010:btrfs_delete_delayed_dir_index+0x28f/0x2a0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel: Code: 8b 43 10 41 b9 ef ff ff ff 4c 89 ea 48 c7 c6 20 1d 45 c0 4c 8b 03 48 8b 88 38 03 00 00 48 8b 04 24 48 8b 78 50 e8 4e 30 02 00 <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 53 48 89 fb e8
Oct 26 16:05:18 black.seblu.net kernel: RSP: 0018:ffff9e5b41a03d40 EFLAGS: 00010286
Oct 26 16:05:18 black.seblu.net kernel: RAX: 0000000000000000 RBX: ffff9742109e2730 RCX: 0000000000000000
Oct 26 16:05:18 black.seblu.net kernel: RDX: 0000000000000000 RSI: ffff974217196388 RDI: ffff974217196388
Oct 26 16:05:18 black.seblu.net kernel: RBP: ffff9740058d1500 R08: 0000000000000000 R09: 000000000000035e
Oct 26 16:05:18 black.seblu.net kernel: R10: ffff9742051e0aa0 R11: 0000000000000001 R12: ffff9742109e2778
Oct 26 16:05:18 black.seblu.net kernel: R13: 000000000000002a R14: ffff9741fd2ff490 R15: ffff974112c43580
Oct 26 16:05:18 black.seblu.net kernel:  __btrfs_unlink_inode+0x280/0x3e0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  btrfs_unlink_inode+0x12/0x40 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  btrfs_unlink+0x87/0xd0 [btrfs]
Oct 26 16:05:18 black.seblu.net kernel:  vfs_unlink+0xea/0x190
Oct 26 16:05:18 black.seblu.net kernel:  do_unlinkat+0x220/0x300
Oct 26 16:05:18 black.seblu.net kernel:  do_syscall_64+0x49/0xf0
Oct 26 16:05:18 black.seblu.net kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Oct 26 16:05:18 black.seblu.net kernel: RIP: 0033:0x7f01a598b32b
Oct 26 16:05:18 black.seblu.net kernel: Code: Bad RIP value.
Oct 26 16:05:18 black.seblu.net kernel: RSP: 002b:00007fffd6de10e8 EFLAGS: 00000206 ORIG_RAX: 0000000000000057
Oct 26 16:05:18 black.seblu.net kernel: RAX: ffffffffffffffda RBX: 000055ef5447c260 RCX: 00007f01a598b32b
Oct 26 16:05:18 black.seblu.net kernel: RDX: 00007fffd6de1120 RSI: 00007fffd6de1120 RDI: 000055ef54b605c0
Oct 26 16:05:18 black.seblu.net kernel: RBP: 000055ef54bd3088 R08: 0000000000000001 R09: 0000000000000000
Oct 26 16:05:18 black.seblu.net kernel: R10: 000000000000015c R11: 0000000000000206 R12: 0000000000000001
Oct 26 16:05:18 black.seblu.net kernel: R13: 000055ef54bd3088 R14: 000055ef54b605c0 R15: 0000000000000000

Comment 1 Seb Lu 2018-11-01 02:30:12 UTC

Some new outputs
# coredumpctl dump         
           PID: 21911 (btrfs)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 11 (SEGV)
     Timestamp: Tue 2018-10-30 11:29:21 CET (1 day 15h ago)
  Command Line: btrfs check -p --repair /dev/sdd
    Executable: /usr/bin/btrfs
 Control Group: /user.slice/user-0.slice/session-c1.scope
          Unit: session-c1.scope
         Slice: user-0.slice
       Session: c1
     Owner UID: 0 (root)
       Boot ID: 5498a2605a454653b31cbadb17d30bac
    Machine ID: f69c0f183dfe4ce2b3c75f3a66a87485
      Hostname: black.seblu.net
       Storage: /var/lib/systemd/coredump/core.btrfs.0.5498a2605a454653b31cbadb17d30bac.21911.1540895361000000.lz4 (truncated)
       Message: Process 21911 (btrfs) of user 0 dumped core.
Refusing to dump core to tty (use shell redirection or specify --output).

Comment 2 Seb Lu 2018-11-02 23:49:25 UTC

Last try:

# btrfs check -p --repair /dev/sdd
enabling repair mode
Opening filesystem to check...
Checking filesystem on /dev/sdd
UUID: d1a00792-7d16-49c3-b79b-7785df31a28d
[1/7] checking root items                      (0:10:41 elapsed, 10587829 items checked))
Fixed 0 roots.
No device size related problem found           (0:41:10 elapsed, 4281626 items checked)
[2/7] checking extents                         (0:41:10 elapsed, 4281626 items checked)
cache and super generation don't match, space cache will be invalidated
[3/7] checking free space cache                (0:00:00 elapsed)
Deleting bad dir index [35889793,96,356] root 5(12:29:07 elapsed, 3865230 items checked)
zsh: segmentation fault (core dumped)  btrfs check -p --repair /dev/sdd43 items checked)

Comment 3 Seb Lu 2018-11-14 10:10:19 UTC

Hello,

Last tries:
1) btrfs check --progress --repair --mode lowmem /dev/sdd
Segfault too after quite long time (~95hours)

# coredumpctl dump
           PID: 4995 (btrfs)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 11 (SEGV)
     Timestamp: Tue 2018-11-13 06:03:30 CET (1 day 5h ago)
  Command Line: btrfs check --progress --repair --mode lowmem /dev/sdd
    Executable: /usr/bin/btrfs
 Control Group: /user.slice/user-0.slice/session-c1.scope
          Unit: session-c1.scope
         Slice: user-0.slice
       Session: c1
     Owner UID: 0 (root)
       Message: Process 4995 (btrfs) of user 0 dumped core.

2) Mount the filesystem with a brand new 4.20-rc2. 

Run a scrub:

# btrfs scrub status /home      
scrub status for d1a00792-7d16-49c3-b79b-7785df31a28d
        scrub started at Wed Nov 14 02:17:37 2018 and finished after 04:41:21
        total bytes scrubbed: 9.90TiB with 0 errors


But the dmesg shows:

nov. 14 01:50:51 black.seblu.net kernel: BTRFS info (device sdd): failed to delete reference to 7f174693f1fde82e3a3a4e689c0446cf, inode 139952 parent 125003654
nov. 14 01:50:51 black.seblu.net kernel: ------------[ cut here ]------------
nov. 14 01:50:51 black.seblu.net kernel: BTRFS: Transaction aborted (error -2)
nov. 14 01:50:51 black.seblu.net kernel: WARNING: CPU: 1 PID: 6502 at fs/btrfs/inode.c:3955 __btrfs_unlink_inode.cold.36+0x9c/0xe5 [btrfs]
nov. 14 01:50:51 black.seblu.net kernel: Modules linked in: tun ip_set_hash_net ip_set nls_cp437 vfat fat intel_rapl x86_pkg_temp_thermal i915 intel_powerclamp kvm_intel kvm irqbypass iTCO_>
nov. 14 01:50:51 black.seblu.net kernel:  xor raid6_pq sd_mod hid_generic usbhid hid ahci libahci libata crc32c_intel xhci_pci xhci_hcd ehci_pci mpt3sas raid_class ehci_hcd scsi_transport_s>
nov. 14 01:50:51 black.seblu.net kernel: CPU: 1 PID: 6502 Comm: rsync_bpc Not tainted 4.20.0-rc2-seblu #1
nov. 14 01:50:51 black.seblu.net kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./H77M-ITX, BIOS P1.90 03/04/2014
nov. 14 01:50:51 black.seblu.net kernel: RIP: 0010:__btrfs_unlink_inode.cold.36+0x9c/0xe5 [btrfs]
nov. 14 01:50:51 black.seblu.net kernel: Code: 48 8b 45 a8 48 8b 50 50 f0 48 0f ba aa 10 0e 00 00 02 8b 45 a0 72 18 83 f8 fb 74 13 89 c6 48 c7 c7 40 ad 6e c0 e8 a5 28 99 c8 <0f> 0b 8b 45 a0>
nov. 14 01:50:51 black.seblu.net kernel: RSP: 0018:ffffbc3702217c00 EFLAGS: 00010286
nov. 14 01:50:51 black.seblu.net kernel: RAX: 0000000000000000 RBX: ffff9e4b8affc470 RCX: 0000000000000006
nov. 14 01:50:51 black.seblu.net kernel: RDX: 0000000000000007 RSI: 0000000000000096 RDI: ffff9e4d17096380
nov. 14 01:50:51 black.seblu.net kernel: RBP: ffffbc3702217c80 R08: 0000000000000001 R09: 0000000000000344
nov. 14 01:50:51 black.seblu.net kernel: R10: 0000000000000004 R11: 0000000000000000 R12: ffff9e4b965587e0
nov. 14 01:50:51 black.seblu.net kernel: R13: ffff9e4b8affc8e0 R14: 00000000000222b0 R15: 0000000007736786
nov. 14 01:50:51 black.seblu.net kernel: FS:  00007f9918128740(0000) GS:ffff9e4d17080000(0000) knlGS:0000000000000000
nov. 14 01:50:51 black.seblu.net kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
nov. 14 01:50:51 black.seblu.net kernel: CR2: 00005596bd6c5e40 CR3: 00000000d41e0002 CR4: 00000000001606e0
nov. 14 01:50:51 black.seblu.net kernel: Call Trace:
nov. 14 01:50:51 black.seblu.net kernel:  btrfs_unlink_inode+0x12/0x40 [btrfs]
nov. 14 01:50:51 black.seblu.net kernel:  btrfs_rename2+0xe4b/0x1bf0 [btrfs]
nov. 14 01:50:51 black.seblu.net kernel:  ? follow_dotdot_rcu+0xa1/0x160
nov. 14 01:50:51 black.seblu.net kernel:  ? inode_permission+0xbb/0x180
nov. 14 01:50:51 black.seblu.net kernel:  ? vfs_rename+0x476/0x870
nov. 14 01:50:51 black.seblu.net kernel:  vfs_rename+0x476/0x870
nov. 14 01:50:51 black.seblu.net kernel:  do_renameat2+0x3df/0x570
nov. 14 01:50:51 black.seblu.net kernel:  __x64_sys_rename+0x17/0x20
nov. 14 01:50:51 black.seblu.net kernel:  do_syscall_64+0x49/0xf0
nov. 14 01:50:51 black.seblu.net kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
nov. 14 01:50:51 black.seblu.net kernel: RIP: 0033:0x7f99181996fb
nov. 14 01:50:51 black.seblu.net kernel: Code: e8 aa fc 07 00 85 c0 0f 95 c0 0f b6 c0 f7 d8 5b c3 66 0f 1f 44 00 00 b8 ff ff ff ff 5b c3 90 f3 0f 1e fa b8 52 00 00 00 0f 05 <48> 3d 00 f0 ff>
nov. 14 01:50:51 black.seblu.net kernel: RSP: 002b:00007ffe89244448 EFLAGS: 00000246 ORIG_RAX: 0000000000000052
nov. 14 01:50:51 black.seblu.net kernel: RAX: ffffffffffffffda RBX: 000055ac646eba00 RCX: 00007f99181996fb
nov. 14 01:50:51 black.seblu.net kernel: RDX: 00007ffe89244460 RSI: 00007ffe892444f0 RDI: 000055ac6470ca98
nov. 14 01:50:51 black.seblu.net kernel: RBP: 00007ffe892444f0 R08: 0000000000000003 R09: 00007ffe89243dc0
nov. 14 01:50:51 black.seblu.net kernel: R10: 00000000000001b0 R11: 0000000000000246 R12: 000055ac6470ca98
nov. 14 01:50:51 black.seblu.net kernel: R13: 00007ffe89244460 R14: 0000000000000000 R15: 00007ffe892464f0
nov. 14 01:50:51 black.seblu.net kernel: ---[ end trace 3609b6d57c1864e4 ]---
nov. 14 01:50:51 black.seblu.net kernel: BTRFS: error (device sdd) in __btrfs_unlink_inode:3955: errno=-2 No such entry
nov. 14 01:50:51 black.seblu.net kernel: BTRFS info (device sdd): forced readonly
nov. 14 01:50:51 black.seblu.net kernel: BTRFS: error (device sdd) in btrfs_rename:9827: errno=-2 No such entry

Comment 4 Seb Lu 2018-11-15 00:41:57 UTC

I tried to re-init the checksum tree. I got this.

# btrfs check --init-csum-tree /dev/sdd
Creating a new CRC tree
Opening filesystem to check...
Checking filesystem on /dev/sdd
UUID: d1a00792-7d16-49c3-b79b-7785df31a28d
Reinitialize checksum tree
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
ref mismatch on [48999001227264 16384] extent item 1, found 0
backref 48999001227264 root 7 not referenced back 0x55d7673d8640
incorrect global backref count on 48999001227264 found 1 wanted 0
backpointer mismatch on [48999001227264 16384]
owner ref check failed [48999001227264 16384]
repair deleting extent record: key [48999001227264,168,16384]
Repaired extent references for 48999001227264
ref mismatch on [48999001522176 16384] extent item 1, found 0
backref 48999001522176 root 7 not referenced back 0x55d7673d9240
incorrect global backref count on 48999001522176 found 1 wanted 0
backpointer mismatch on [48999001522176 16384]
owner ref check failed [48999001522176 16384]
repair deleting extent record: key [48999001522176,168,16384]
Failed to find [49039413870592, 168, 16384]
btrfs unable to find ref byte nr 49039413870592 parent 0 root 2  owner 0 offset 0
transaction.c:195: btrfs_commit_transaction: BUG_ON `ret` triggered, value -5
btrfs(+0x46c37)[0x55d6f0607c37]
btrfs(btrfs_commit_transaction+0x1be)[0x55d6f060825e]
btrfs(+0x67f9c)[0x55d6f0628f9c]
btrfs(cmd_check+0x1384)[0x55d6f062ae04]
btrfs(main+0x88)[0x55d6f05d40b8]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0x7f3b53d95223]
btrfs(_start+0x2e)[0x55d6f05d41de]
zsh: abort (core dumped)  btrfs check --init-csum-tree /dev/sdd

Do you have some guidance to fix this?

Comment 5 Seb Lu 2018-11-18 19:33:06 UTC

Now, each time I try to access to a specific file inside the filesystem, it is remounted read-only and a lot of error are written in dmesg.

nov. 17 19:39:23 black.seblu.net kernel: BTRFS info (device sdd): using free space tree
nov. 17 19:39:26 black.seblu.net kernel: BTRFS info (device sdd): bdev /dev/sdd errs: wr 2, rd 5, flush 0, corrupt 113, gen 5
nov. 17 19:39:26 black.seblu.net kernel: BTRFS info (device sdd): bdev /dev/sde errs: wr 1034, rd 0, flush 0, corrupt 0, gen 0
nov. 17 19:39:26 black.seblu.net kernel: BTRFS info (device sdd): bdev /dev/sdb errs: wr 0, rd 0, flush 0, corrupt 177280, gen 0
nov. 17 19:39:26 black.seblu.net kernel: BTRFS info (device sdd): bdev /dev/sdf errs: wr 533556, rd 646011, flush 0, corrupt 366, gen 14
nov. 17 19:39:26 black.seblu.net kernel: BTRFS info (device sdd): checking UUID tree
nov. 17 19:39:56 black.seblu.net kernel: BTRFS error (device sdd): incorrect extent count for 48998995984384; counted 1507, expected 42>
nov. 17 19:39:56 black.seblu.net kernel: WARNING: CPU: 1 PID: 713 at fs/btrfs/extent-tree.c:926 btrfs_lookup_extent_info+0x3bc/0x3d0 [b>
nov. 17 19:39:56 black.seblu.net kernel: Modules linked in: tun ip_set_hash_net ip_set nfnetlink nls_cp437 vfat fat intel_rapl x86_pkg_>
nov. 17 19:39:56 black.seblu.net kernel: CPU: 1 PID: 713 Comm: rm Not tainted 4.18.18-seblu #1
nov. 17 19:39:56 black.seblu.net kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./H77M-ITX, BIOS P1.90 03/04/2014
nov. 17 19:39:56 black.seblu.net kernel: RIP: 0010:btrfs_lookup_extent_info+0x3bc/0x3d0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel: Code: 75 35 48 83 c4 68 5b 5d 41 5c 41 5d 41 5e 41 5f c3 80 48 6a 14 e9 cc fc ff ff 0f 0b 48 8>
nov. 17 19:39:56 black.seblu.net kernel: RSP: 0018:ffff96674162baf0 EFLAGS: 00010246
nov. 17 19:39:56 black.seblu.net kernel: RAX: 0000000000000000 RBX: ffff886050a49d90 RCX: ffff88604bdc3130
nov. 17 19:39:56 black.seblu.net kernel: RDX: 0000000000000000 RSI: 00002c9616b50000 RDI: ffff886026158d60
nov. 17 19:39:56 black.seblu.net kernel: RBP: ffff886026158c00 R08: 0000000000000000 R09: 00002c9413c04000
nov. 17 19:39:56 black.seblu.net kernel: R10: 000000000000057d R11: 0000000000000001 R12: ffff8860564f59c0
nov. 17 19:39:56 black.seblu.net kernel: R13: ffff886026158d60 R14: 0000000000000000 R15: 00002c9616b50000
nov. 17 19:39:56 black.seblu.net kernel: FS:  00007f112305e540(0000) GS:ffff88605f280000(0000) knlGS:0000000000000000
nov. 17 19:39:56 black.seblu.net kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
nov. 17 19:39:56 black.seblu.net kernel: CR2: 0000564537742168 CR3: 00000001e5670002 CR4: 00000000001606e0
nov. 17 19:39:56 black.seblu.net kernel: Call Trace:
nov. 17 19:39:56 black.seblu.net kernel:  update_ref_for_cow+0x15b/0x300 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  __btrfs_cow_block+0x236/0x4d0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_cow_block+0xd3/0x110 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_search_slot+0x283/0xa20 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_lookup_dir_item+0x77/0xc0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  __btrfs_unlink_inode+0xa3/0x3e0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  ? start_transaction+0x97/0x370 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_unlink_inode+0x12/0x40 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_unlink+0x87/0xd0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  vfs_unlink+0x104/0x1a0
nov. 17 19:39:56 black.seblu.net kernel:  do_unlinkat+0x234/0x320
nov. 17 19:39:56 black.seblu.net kernel:  do_syscall_64+0x49/0xf0
nov. 17 19:39:56 black.seblu.net kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
nov. 17 19:39:56 black.seblu.net kernel: RIP: 0033:0x7f1122f8735b
nov. 17 19:39:56 black.seblu.net kernel: Code: 73 01 c3 48 8b 0d 05 fb 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 0>
nov. 17 19:39:56 black.seblu.net kernel: RSP: 002b:00007ffd5def4d38 EFLAGS: 00000246 ORIG_RAX: 0000000000000107
nov. 17 19:39:56 black.seblu.net kernel: RAX: ffffffffffffffda RBX: 0000563e4f301770 RCX: 00007f1122f8735b
nov. 17 19:39:56 black.seblu.net kernel: RDX: 0000000000000000 RSI: 0000563e4f300540 RDI: 00000000ffffff9c
nov. 17 19:39:56 black.seblu.net kernel: RBP: 0000563e4f3004b0 R08: 0000000000000003 R09: 0000000000000000
nov. 17 19:39:56 black.seblu.net kernel: R10: 0000ffff00000000 R11: 0000000000000246 R12: 00007ffd5def4f20
nov. 17 19:39:56 black.seblu.net kernel: R13: 0000000000000000 R14: 0000563e4f301770 R15: 0000000000000000
nov. 17 19:39:56 black.seblu.net kernel: ---[ end trace 57db90cd911170be ]---
nov. 17 19:39:56 black.seblu.net kernel: BTRFS: error (device sdd) in update_ref_for_cow:937: errno=-30 Readonly filesystem
nov. 17 19:39:56 black.seblu.net kernel: BTRFS info (device sdd): forced readonly
nov. 17 19:39:56 black.seblu.net kernel: ------------[ cut here ]------------
nov. 17 19:39:56 black.seblu.net kernel: BTRFS: Transaction aborted (error -30)
nov. 17 19:39:56 black.seblu.net kernel: WARNING: CPU: 1 PID: 713 at fs/btrfs/ctree.c:1080 __btrfs_cow_block+0x482/0x4d0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel: Modules linked in: tun ip_set_hash_net ip_set nfnetlink nls_cp437 vfat fat intel_rapl x86_pkg_>
nov. 17 19:39:56 black.seblu.net kernel: CPU: 1 PID: 713 Comm: rm Tainted: G        W         4.18.18-seblu #1
nov. 17 19:39:56 black.seblu.net kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./H77M-ITX, BIOS P1.90 03/04/2014
nov. 17 19:39:56 black.seblu.net kernel: RIP: 0010:__btrfs_cow_block+0x482/0x4d0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel: Code: a8 08 0e 00 00 02 0f 92 c0 41 83 fe fb 0f 84 c9 ef 09 00 84 c0 0f 85 c1 ef 09 00 44 89 f>
nov. 17 19:39:56 black.seblu.net kernel: RSP: 0018:ffff96674162bbe8 EFLAGS: 00010282
nov. 17 19:39:56 black.seblu.net kernel: RAX: 0000000000000000 RBX: ffff886054914000 RCX: 0000000000000006
nov. 17 19:39:56 black.seblu.net kernel: RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff88605f296310
nov. 17 19:39:56 black.seblu.net kernel: RBP: ffff886054522000 R08: 0000000000000001 R09: 0000000000000369
nov. 17 19:39:56 black.seblu.net kernel: R10: 0000000000000004 R11: 0000000000000000 R12: ffff8860564f59c0
nov. 17 19:39:56 black.seblu.net kernel: R13: ffff886054761388 R14: 00000000ffffffe2 R15: ffff886054761040
nov. 17 19:39:56 black.seblu.net kernel: FS:  00007f112305e540(0000) GS:ffff88605f280000(0000) knlGS:0000000000000000
nov. 17 19:39:56 black.seblu.net kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
nov. 17 19:39:56 black.seblu.net kernel: CR2: 0000564537742168 CR3: 00000001e5670002 CR4: 00000000001606e0
nov. 17 19:39:56 black.seblu.net kernel: Call Trace:
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_cow_block+0xd3/0x110 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_search_slot+0x283/0xa20 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_lookup_dir_item+0x77/0xc0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  __btrfs_unlink_inode+0xa3/0x3e0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  ? start_transaction+0x97/0x370 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_unlink_inode+0x12/0x40 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  btrfs_unlink+0x87/0xd0 [btrfs]
nov. 17 19:39:56 black.seblu.net kernel:  vfs_unlink+0x104/0x1a0
nov. 17 19:39:56 black.seblu.net kernel:  do_unlinkat+0x234/0x320
nov. 17 19:39:56 black.seblu.net kernel:  do_syscall_64+0x49/0xf0
nov. 17 19:39:56 black.seblu.net kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
nov. 17 19:39:56 black.seblu.net kernel: RIP: 0033:0x7f1122f8735b
nov. 17 19:39:56 black.seblu.net kernel: Code: 73 01 c3 48 8b 0d 05 fb 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 0>
nov. 17 19:39:56 black.seblu.net kernel: RSP: 002b:00007ffd5def4d38 EFLAGS: 00000246 ORIG_RAX: 0000000000000107
nov. 17 19:39:56 black.seblu.net kernel: RAX: ffffffffffffffda RBX: 0000563e4f301770 RCX: 00007f1122f8735b
nov. 17 19:39:56 black.seblu.net kernel: RDX: 0000000000000000 RSI: 0000563e4f300540 RDI: 00000000ffffff9c
nov. 17 19:39:56 black.seblu.net kernel: RBP: 0000563e4f3004b0 R08: 0000000000000003 R09: 0000000000000000
nov. 17 19:39:56 black.seblu.net kernel: R10: 0000ffff00000000 R11: 0000000000000246 R12: 00007ffd5def4f20
nov. 17 19:39:56 black.seblu.net kernel: R13: 0000000000000000 R14: 0000563e4f301770 R15: 0000000000000000
nov. 17 19:39:56 black.seblu.net kernel: ---[ end trace 57db90cd911170bf ]---
nov. 17 19:39:56 black.seblu.net kernel: BTRFS error (device sdd): incorrect extent count for 49000103280640; counted 1955, expected 19>
nov. 17 19:40:07 black.seblu.net kernel: BTRFS error (device sdd): cleaner transaction attach returned -30
nov. 17 19:40:07 black.seblu.net kernel: BTRFS warning (device sdd): page private not zero on page 49014498148352
nov. 17 19:40:07 black.seblu.net kernel: BTRFS warning (device sdd): page private not zero on page 49014498152448
nov. 17 19:40:07 black.seblu.net kernel: BTRFS warning (device sdd): page private not zero on page 49014498156544
nov. 17 19:40:07 black.seblu.net kernel: BTRFS warning (device sdd): page private not zero on page 49014498160640

Note You need to log in before you can comment on or make changes to this bug.