Bug 197587

Summary: kernel BUG at fs/btrfs/ctree.h:3457!
Product: File System Reporter: lakshmipathi (lakshmipathi.g)
Component: btrfsAssignee: Josef Bacik (josef)
Status: RESOLVED CODE_FIX    
Severity: normal    
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 4.14.0-rc6 Subsystem:
Regression: No Bisected commit-id:
Attachments: fsck-tests

Description lakshmipathi 2017-10-29 10:07:59 UTC
While running btrfs-progs (commit a7a1ea0f4f2a1d6eeeb3d106e062c7f1034f16d4) fsck-tests failed see attached logs. Then tried starting xfstests during this time found below kernel log:

[  311.255182] BTRFS: device fsid 66387dab-dd02-478d-8ea7-5e6f106cfe36 devid 1 transid 5 /dev/xvdb
[  311.437634] BTRFS info (device xvdb): disk space caching is enabled
[  311.440634] BTRFS info (device xvdb): has skinny extents
[  311.443331] BTRFS info (device xvdb): flagging fs with big metadata feature
[  311.449820] BTRFS info (device xvdb): enabling ssd optimizations
[  311.454223] BTRFS info (device xvdb): creating UUID tree
[  311.463353] BTRFS critical (device xvdb): corrupt leaf: root=1 block=21037056 slot=10 ino=256 file_offset=0, invalid disk_num_bytes for file extent, have 6, should be aligned to 4096
[  311.470698] BTRFS info (device xvdb): leaf 21037056 total ptrs 13 free space 12844
[  311.473958] 	item 0 key (2 132 0) itemoff 15844 itemsize 439
[  311.476447] 		root data bytenr 20971520 refs 1
[  311.478339] 	item 1 key (4 132 0) itemoff 15405 itemsize 439
[  311.480784] 		root data bytenr 20987904 refs 1
[  311.482673] 	item 2 key (5 12 6) itemoff 15388 itemsize 17
[  311.485056] 	item 3 key (5 132 0) itemoff 14949 itemsize 439
[  311.487614] 		root data bytenr 4194304 refs 1
[  311.489563] 	item 4 key (6 1 0) itemoff 14789 itemsize 160
[  311.492091] 		inode generation 3 size 0 mode 40755
[  311.494171] 	item 5 key (6 12 6) itemoff 14777 itemsize 12
[  311.496674] 	item 6 key (6 84 2378154706) itemoff 14740 itemsize 37
[  311.499719] 		dir oid 5 type 2
[  311.501089] 	item 7 key (7 132 0) itemoff 14301 itemsize 439
[  311.503537] 		root data bytenr 4325376 refs 1
[  311.505434] 	item 8 key (9 132 0) itemoff 13862 itemsize 439
[  311.507847] 		root data bytenr 21020672 refs 1
[  311.509879] 	item 9 key (256 1 0) itemoff 13702 itemsize 160
[  311.512242] 		inode generation 0 size 0 mode 100600
[  311.514227] 	item 10 key (256 108 0) itemoff 13649 itemsize 53
[  311.516436] 		extent data disk bytenr 0 nr 6
[  311.518078] 		extent data offset 2 nr 0 ram 1099511627776
[  311.520125] 	item 11 key (18446744073709551605 0 20971520) itemoff 13608 itemsize 41
[  311.523052] 	item 12 key (18446744073709551607 132 0) itemoff 13169 itemsize 439
[  311.525859] 		root data bytenr 4341760 refs 1
[  311.527516] assertion failed: 0, file: fs/btrfs/disk-io.c, line: 3853
[  311.529933] ------------[ cut here ]------------
[  311.531743] kernel BUG at fs/btrfs/ctree.h:3457!
[  311.533584] invalid opcode: 0000 [#1] SMP
[  311.535131] Modules linked in: isofs cirrus drm_kms_helper ttm intel_rapl sb_edac drm crct10dif_pclmul crc32_pclmul crc32c_intel ppdev ghash_clmulni_intel xen_netfront intel_rapl_perf parport_pc parport i2c_piix4 xen_blkfront serio_raw ata_generic pata_acpi
[  311.543766] CPU: 1 PID: 4511 Comm: mount Not tainted 4.14.0-rc6 #1
[  311.546103] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
[  311.548478] task: ffff8fcb7b034c00 task.stack: ffffa9978467c000
[  311.550734] RIP: 0010:assfail.constprop.49+0x1c/0x22
[  311.553831] RSP: 0018:ffffa9978467f548 EFLAGS: 00010286
[  311.557050] RAX: 0000000000000039 RBX: ffff8fcb7a525a98 RCX: 0000000000000000
[  311.561128] RDX: 0000000000000000 RSI: ffff8fcb7fe4e0b8 RDI: ffff8fcb7fe4e0b8
[  311.565148] RBP: ffffa9978467f548 R08: 00000000000002d8 R09: 0000000000000000
[  311.569067] R10: 0000000000000003 R11: 0000000000000000 R12: ffff8fcb7bc78000
[  311.573051] R13: ffff8fcb7b4c0000 R14: 0000000000000007 R15: ffffa9978467f6e6
[  311.576981] FS:  00007ffad5b59480(0000) GS:ffff8fcb7fe40000(0000) knlGS:0000000000000000
[  311.581264] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  311.584711] CR2: 00007f31592e7710 CR3: 000000043b8e2006 CR4: 00000000001606e0
[  311.588607] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  311.592463] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  311.596338] Call Trace:
[  311.598461]  btrfs_mark_buffer_dirty+0xc9/0x100
[  311.601301]  setup_items_for_insert+0x1e0/0x3f0
[  311.604172]  __btrfs_drop_extents+0xab6/0xe60
[  311.606924]  insert_reserved_file_extent.constprop.69+0x8d/0x2d0
[  311.610356]  __btrfs_prealloc_file_range+0x15f/0x480
[  311.613346]  ? btrfs_free_path.part.28+0x21/0x30
[  311.616182]  btrfs_prealloc_file_range_trans+0x23/0x25
[  311.619230]  cache_save_setup+0x268/0x3a0
[  311.621844]  btrfs_start_dirty_block_groups+0x1ed/0x480
[  311.625309]  btrfs_commit_transaction+0x109/0x950
[  311.628305]  ? btrfs_create_tree+0x280/0x290
[  311.631013]  btrfs_create_uuid_tree+0x58/0x160
[  311.633798]  open_ctree+0x2289/0x24df
[  311.636283]  btrfs_mount+0xe16/0xefc
[  311.638723]  ? find_next_bit+0xb/0x10
[  311.641187]  ? cpumask_next+0x1b/0x20
[  311.643650]  mount_fs+0x32/0x150
[  311.645941]  ? __alloc_percpu+0x15/0x20
[  311.648481]  vfs_kern_mount.part.22+0x5d/0x120
[  311.651284]  vfs_kern_mount+0x13/0x20
[  311.653677]  btrfs_mount+0x1a8/0xefc
[  311.656040]  ? find_next_bit+0xb/0x10
[  311.658479]  mount_fs+0x32/0x150
[  311.661341]  ? __alloc_percpu+0x15/0x20
[  311.663873]  vfs_kern_mount.part.22+0x5d/0x120
[  311.666673]  do_mount+0x5c7/0xca0
[  311.668942]  ? kmem_cache_alloc_trace+0x165/0x1d0
[  311.671705]  ? copy_mount_options+0x2c/0x220
[  311.674424]  SyS_mount+0x98/0xe0
[  311.676651]  entry_SYSCALL_64_fastpath+0x1a/0xa5
[  311.679384] RIP: 0033:0x7ffad4bad28a
[  311.681625] RSP: 002b:00007fffdc2df2c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
[  311.685346] RAX: ffffffffffffffda RBX: 00007ffad572d7ef RCX: 00007ffad4bad28a
[  311.688904] RDX: 000056491a0792f0 RSI: 000056491a079330 RDI: 000056491a079310
[  311.692470] RBP: 00007ffad593f184 R08: 000056491a079290 R09: 00007fffdc2de1c8
[  311.696038] R10: 00000000c0ed0000 R11: 0000000000000246 R12: 000056491a079140
[  311.699605] R13: 00007fffdc2df5e8 R14: 00005649195b94a0 R15: 00000000ffffffff
[  311.703114] Code: 40 70 31 d2 48 89 86 88 00 00 00 89 d0 5d c3 55 89 f1 48 c7 c2 bb 73 cd a0 48 89 fe 48 c7 c7 80 49 cd a0 48 89 e5 e8 47 03 d5 ff <0f> 0b 90 90 90 90 0f 1f 44 00 00 48 8b 46 20 a8 02 0f 84 a3 00 
[  311.712171] RIP: assfail.constprop.49+0x1c/0x22 RSP: ffffa9978467f548
[  311.715655] ---[ end trace ec5f9265788cdc5e ]---



-bash-4.4$ uname -a
Linux ip-172-31-46-165.us-west-2.compute.internal 4.14.0-rc6 #1 SMP Sun Oct 29 09:40:11 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

installed from https://github.com/kdave/btrfs-devel/archive/misc-next.zip
Comment 1 lakshmipathi 2017-10-29 10:09:24 UTC
Created attachment 260425 [details]
fsck-tests
Comment 2 lakshmipathi 2017-10-29 10:11:09 UTC
xfstests error message:
 ./common/rc: line 192:  4511 Segmentation fault      $MOUNT_PROG `_mount_ops_filter $*`
 out: common/rc: retrying test device mount with external set
Comment 3 lakshmipathi 2017-10-29 10:25:42 UTC
Tried manual mount it also hangs:

$sudo mount /dev/xvdb /mnt
^C^Z^C^C^Z^Z^Z^Z^C^C^Z

^Z^C^Z^C^C
Comment 4 lakshmipathi 2017-11-03 06:33:59 UTC
After apply this patch from Qu https://patchwork.kernel.org/patch/10038011/ . This issue has been resolved.
Comment 5 lakshmipathi 2017-11-06 05:21:23 UTC
Nov 06 05:08:09 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): disk space caching is enabled
Nov 06 05:08:09 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): has skinny extents
Nov 06 05:08:09 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): flagging fs with big metadata feature
Nov 06 05:08:09 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): enabling ssd optimizations
Nov 06 05:08:09 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): creating UUID tree
Nov 06 05:08:10 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): dev_replace from /dev/xvdd (devid 2) to /dev/xvdg started
Nov 06 05:08:12 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): dev_replace from /dev/xvdd (devid 2) to /dev/xvdg finished
Nov 06 05:08:16 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS: device fsid 3ebc32b5-ba0a-405b-8acf-4785b0a4b81a devid 1 transid 5 /dev/xvdc
Nov 06 05:08:16 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS: device fsid 3ebc32b5-ba0a-405b-8acf-4785b0a4b81a devid 2 transid 5 /dev/xvdd
Nov 06 05:08:17 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS: device fsid 3ebc32b5-ba0a-405b-8acf-4785b0a4b81a devid 3 transid 5 /dev/xvde
Nov 06 05:08:17 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS: device fsid 3ebc32b5-ba0a-405b-8acf-4785b0a4b81a devid 4 transid 5 /dev/xvdf
Nov 06 05:08:17 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): disk space caching is enabled
Nov 06 05:08:17 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): has skinny extents
Nov 06 05:08:17 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): flagging fs with big metadata feature
Nov 06 05:08:17 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): enabling ssd optimizations
Nov 06 05:08:17 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): creating UUID tree
Nov 06 05:08:18 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): dev_replace from /dev/xvdd (devid 2) to /dev/xvdg started
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS critical (device xvdf): corrupt leaf: root=9 block=4400742400 slot=0, invalid nritems, have 0 should not be 0 for non-root leaf
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: BTRFS info (device xvdf): leaf 4400742400 total ptrs 0 free space 16283
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: assertion failed: 0, file: fs/btrfs/disk-io.c, line: 540
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: ------------[ cut here ]------------
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: kernel BUG at fs/btrfs/ctree.h:3457!
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: invalid opcode: 0000 [#1] SMP
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: Modules linked in: cirrus drm_kms_helper ttm drm intel_rapl sb_edac ppdev crct10dif_pclmul crc32_pclmul crc32c_intel xen_netfront ghash_clmulni_intel intel_rapl_perf parport_pc parport i2c_piix4 xen_blkfront serio_raw ata_generic pata_acpi
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: CPU: 1 PID: 13413 Comm: fsstress Not tainted 4.14.0-rc7 #1
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: Hardware name: Xen HVM domU, BIOS 4.2.amazon 12/12/2016
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: task: ffff9d2331a48000 task.stack: ffffbc0f43fd4000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RIP: 0010:assfail.constprop.49+0x1c/0x22
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RSP: 0018:ffffbc0f43fd79c0 EFLAGS: 00010282
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RAX: 0000000000000038 RBX: ffff9d2335078000 RCX: 0000000000000000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RDX: 0000000000000000 RSI: ffff9d233fe4e0b8 RDI: ffff9d233fe4e0b8
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RBP: ffffbc0f43fd79c0 R08: 00000000000003f5 R09: 0000000000000000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: R10: ffffbc0f43fd7938 R11: 0000000000000000 R12: 0000000000000000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: R13: ffff9d23316f1bc0 R14: ffff9d2335813400 R15: ffff9d2335bd0000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: FS:  00007f75051fde80(0000) GS:ffff9d233fe40000(0000) knlGS:0000000000000000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: CR2: 00007f750509b000 CR3: 000000043bc7b006 CR4: 00000000001606e0
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: Call Trace:
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btree_csum_one_bio+0x14b/0x180
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btree_submit_bio_hook+0x43/0xc0
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  submit_one_bio+0x5e/0x90
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  flush_epd_write_bio.isra.41+0x21/0x40
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  flush_write_bio+0xe/0x10
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btree_write_cache_pages+0x2e8/0x420
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btree_writepages+0x5d/0x70
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  do_writepages+0x48/0xe0
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  ? _cond_resched+0x22/0x40
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  ? merge_state.part.46+0x44/0x130
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  __filemap_fdatawrite_range+0xc1/0x100
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  ? __filemap_fdatawrite_range+0xc1/0x100
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  ? free_extent_state+0x98/0x100
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  filemap_fdatawrite_range+0x13/0x20
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btrfs_write_marked_extents+0x68/0x140
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btrfs_write_and_wait_marked_extents.constprop.21+0x4f/0x90
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btrfs_commit_transaction+0x6ad/0x950
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  ? btrfs_commit_transaction+0x6ad/0x950
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  ? finish_wait+0x80/0x80
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  btrfs_sync_file+0x328/0x400
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  vfs_fsync_range+0x49/0xa0
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  do_fsync+0x3d/0x70
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  SyS_fsync+0x10/0x20
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel:  entry_SYSCALL_64_fastpath+0x1a/0xa5
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RIP: 0033:0x7f75049cf720
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RSP: 002b:00007fff253c0158 EFLAGS: 00000246 ORIG_RAX: 000000000000004a
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RAX: ffffffffffffffda RBX: 0000000000000037 RCX: 00007f75049cf720
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RDX: 00007fff253c00c0 RSI: 00007fff253c00c0 RDI: 0000000000000003
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RBP: 0000000000007338 R08: 0000000001cd0160 R09: 00007fff253c016c
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: R10: 0000000000000006 R11: 0000000000000246 R12: 0000000000000003
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: R13: 00000000000e04b9 R14: 0000000001cd0c00 R15: 0000000000000000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: Code: 40 70 31 d2 48 89 86 88 00 00 00 89 d0 5d c3 55 89 f1 48 c7 c2 db f7 cd bc 48 89 fe 48 c7 c7 a0 cd cd bc 48 89 e5 e8 e7 18 d5 ff <0f> 0b 90 90 90 90 0f 1f 44 00 00 48 8b 46 20 a8 02 0f 84 a3 00 
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RIP: assfail.constprop.49+0x1c/0x22 RSP: ffffbc0f43fd79c0
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: ---[ end trace f9e6a562c32ec923 ]---
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: ------------[ cut here ]------------
1Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: WARNING: CPU: 1 PID: 13413 at kernel/exit.c:771 do_exit+0x51/0xb40
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: Modules linked in: cirrus drm_kms_helper ttm drm intel_rapl sb_edac ppdev crct10dif_pclmul crc32_pclmul crc32c_intel xen_netfront ghash_clmulni_intel intel_rapl_perf parport_pc parport i2c_piix4 xen_blkfront serio_raw ata_generic pata_acpi
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: CPU: 1 PID: 13413 Comm: fsstress Tainted: G      D         4.14.0-rc7 #1
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: Hardware name: Xen HVM domU, BIOS 4.2.amazon 12/12/2016
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: task: ffff9d2331a48000 task.stack: ffffbc0f43fd4000
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RIP: 0010:do_exit+0x51/0xb40
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RSP: 0018:ffffbc0f43fd7ed8 EFLAGS: 00010206
Nov 06 05:08:23 ip-172-31-32-145.us-west-2.compute.internal kernel: RAX: ffffbc0f43fd7d40 RBX: ffff9d2331a48000 RCX: ffffbc0f43fd7d50
Comment 6 lakshmipathi 2017-11-06 05:23:26 UTC
pls ignore previous output: here is something little formatted:
Nov 06 05:08:18 ec2 kernel: BTRFS info (device xvdf): dev_replace from /dev/xvdd (devid 2) to /dev/xvdg started
Nov 06 05:08:23 ec2 kernel: BTRFS critical (device xvdf): corrupt leaf: root=9 block=4400742400 slot=0, invalid nritems, have 0 should not be 0 for non-root leaf
Nov 06 05:08:23 ec2 kernel: BTRFS info (device xvdf): leaf 4400742400 total ptrs 0 free space 16283
Nov 06 05:08:23 ec2 kernel: assertion failed: 0, file: fs/btrfs/disk-io.c, line: 540
Nov 06 05:08:23 ec2 kernel: ------------[ cut here ]------------
Nov 06 05:08:23 ec2 kernel: kernel BUG at fs/btrfs/ctree.h:3457!
Nov 06 05:08:23 ec2 kernel: invalid opcode: 0000 [#1] SMP
Nov 06 05:08:23 ec2 kernel: Modules linked in: cirrus drm_kms_helper ttm drm intel_rapl sb_edac ppdev crct10dif_pclmul crc32_pclmul crc32c_intel xen_netfront ghash_clmulni_intel intel_rapl_perf parport_pc parport i2c_piix4 xen_blkfront serio_raw ata_generic pata_acpi
Nov 06 05:08:23 ec2 kernel: CPU: 1 PID: 13413 Comm: fsstress Not tainted 4.14.0-rc7 #1
Nov 06 05:08:23 ec2 kernel: Hardware name: Xen HVM domU, BIOS 4.2.amazon 12/12/2016
Nov 06 05:08:23 ec2 kernel: task: ffff9d2331a48000 task.stack: ffffbc0f43fd4000
Nov 06 05:08:23 ec2 kernel: RIP: 0010:assfail.constprop.49+0x1c/0x22
Nov 06 05:08:23 ec2 kernel: RSP: 0018:ffffbc0f43fd79c0 EFLAGS: 00010282
Nov 06 05:08:23 ec2 kernel: RAX: 0000000000000038 RBX: ffff9d2335078000 RCX: 0000000000000000
Nov 06 05:08:23 ec2 kernel: RDX: 0000000000000000 RSI: ffff9d233fe4e0b8 RDI: ffff9d233fe4e0b8
Nov 06 05:08:23 ec2 kernel: RBP: ffffbc0f43fd79c0 R08: 00000000000003f5 R09: 0000000000000000
Nov 06 05:08:23 ec2 kernel: R10: ffffbc0f43fd7938 R11: 0000000000000000 R12: 0000000000000000
Nov 06 05:08:23 ec2 kernel: R13: ffff9d23316f1bc0 R14: ffff9d2335813400 R15: ffff9d2335bd0000
Nov 06 05:08:23 ec2 kernel: FS:  00007f75051fde80(0000) GS:ffff9d233fe40000(0000) knlGS:0000000000000000
Nov 06 05:08:23 ec2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 06 05:08:23 ec2 kernel: CR2: 00007f750509b000 CR3: 000000043bc7b006 CR4: 00000000001606e0
Nov 06 05:08:23 ec2 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 06 05:08:23 ec2 kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 06 05:08:23 ec2 kernel: Call Trace:
Nov 06 05:08:23 ec2 kernel:  btree_csum_one_bio+0x14b/0x180
Nov 06 05:08:23 ec2 kernel:  btree_submit_bio_hook+0x43/0xc0
Nov 06 05:08:23 ec2 kernel:  submit_one_bio+0x5e/0x90
Nov 06 05:08:23 ec2 kernel:  flush_epd_write_bio.isra.41+0x21/0x40
Nov 06 05:08:23 ec2 kernel:  flush_write_bio+0xe/0x10
Nov 06 05:08:23 ec2 kernel:  btree_write_cache_pages+0x2e8/0x420
Nov 06 05:08:23 ec2 kernel:  btree_writepages+0x5d/0x70
Nov 06 05:08:23 ec2 kernel:  do_writepages+0x48/0xe0
Nov 06 05:08:23 ec2 kernel:  ? _cond_resched+0x22/0x40
Nov 06 05:08:23 ec2 kernel:  ? merge_state.part.46+0x44/0x130
Nov 06 05:08:23 ec2 kernel:  __filemap_fdatawrite_range+0xc1/0x100
Nov 06 05:08:23 ec2 kernel:  ? __filemap_fdatawrite_range+0xc1/0x100
Nov 06 05:08:23 ec2 kernel:  ? free_extent_state+0x98/0x100
Nov 06 05:08:23 ec2 kernel:  filemap_fdatawrite_range+0x13/0x20
Nov 06 05:08:23 ec2 kernel:  btrfs_write_marked_extents+0x68/0x140
Nov 06 05:08:23 ec2 kernel:  btrfs_write_and_wait_marked_extents.constprop.21+0x4f/0x90
Nov 06 05:08:23 ec2 kernel:  btrfs_commit_transaction+0x6ad/0x950
Nov 06 05:08:23 ec2 kernel:  ? btrfs_commit_transaction+0x6ad/0x950
Nov 06 05:08:23 ec2 kernel:  ? finish_wait+0x80/0x80
Nov 06 05:08:23 ec2 kernel:  btrfs_sync_file+0x328/0x400
Nov 06 05:08:23 ec2 kernel:  vfs_fsync_range+0x49/0xa0
Nov 06 05:08:23 ec2 kernel:  do_fsync+0x3d/0x70
Nov 06 05:08:23 ec2 kernel:  SyS_fsync+0x10/0x20
Nov 06 05:08:23 ec2 kernel:  entry_SYSCALL_64_fastpath+0x1a/0xa5
Nov 06 05:08:23 ec2 kernel: RIP: 0033:0x7f75049cf720
Nov 06 05:08:23 ec2 kernel: RSP: 002b:00007fff253c0158 EFLAGS: 00000246 ORIG_RAX: 000000000000004a
Nov 06 05:08:23 ec2 kernel: RAX: ffffffffffffffda RBX: 0000000000000037 RCX: 00007f75049cf720
Nov 06 05:08:23 ec2 kernel: RDX: 00007fff253c00c0 RSI: 00007fff253c00c0 RDI: 0000000000000003
Nov 06 05:08:23 ec2 kernel: RBP: 0000000000007338 R08: 0000000001cd0160 R09: 00007fff253c016c
Nov 06 05:08:23 ec2 kernel: R10: 0000000000000006 R11: 0000000000000246 R12: 0000000000000003
Nov 06 05:08:23 ec2 kernel: R13: 00000000000e04b9 R14: 0000000001cd0c00 R15: 0000000000000000
Nov 06 05:08:23 ec2 kernel: Code: 40 70 31 d2 48 89 86 88 00 00 00 89 d0 5d c3 55 89 f1 48 c7 c2 db f7 cd bc 48 89 fe 48 c7 c7 a0 cd cd bc 48 89 e5 e8 e7 18 d5 ff <0f> 0b 90 90 90 90 0f 1f 44 00 00 48 8b 46 20 a8 02 0f 84 a3 00 
Nov 06 05:08:23 ec2 kernel: RIP: assfail.constprop.49+0x1c/0x22 RSP: ffffbc0f43fd79c0
Nov 06 05:08:23 ec2 kernel: ---[ end trace f9e6a562c32ec923 ]---
Nov 06 05:08:23 ec2 kernel: ------------[ cut here ]------------
1Nov 06 05:08:23 ec2 kernel: WARNING: CPU: 1 PID: 13413 at kernel/exit.c:771 do_exit+0x51/0xb40
Comment 7 lakshmipathi 2017-11-27 03:52:50 UTC
Seems like Qu patches resolved this. Will run test for more days and close this bug.