Bug 196709

Summary: WARNING: CPU: 0 PID: 2524 at fs/nfs/pagelist.c:888 nfs_pageio_add_request+0x265/0x280 [nfs]
Product: File System Reporter: JianhongYin (yin-jianhong)
Component: NFSAssignee: Trond Myklebust (trondmy)
Status: RESOLVED CODE_FIX    
Severity: normal CC: trondmy, yin-jianhong
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 4.13 rc5 Subsystem:
Regression: No Bisected commit-id:
Attachments: NFSv4: Fix up mirror allocation

Description JianhongYin 2017-08-19 07:52:17 UTC 
Get panic when run "xfstests generic/010"
  (https://github.com/kdave/xfstests)

[31508.475784] run fstests generic/008 at 2017-08-18 13:19:13 
[31537.646608] run fstests generic/009 at 2017-08-18 13:19:42 
[-- MARK -- Fri Aug 18 17:20:00 2017] 
[31567.023736] run fstests generic/010 at 2017-08-18 13:20:12 
[31584.540076] ------------[ cut here ]------------ 
[31584.543667] WARNING: CPU: 0 PID: 2524 at fs/nfs/pagelist.c:888 nfs_pageio_add_request+0x265/0x280 [nfs] 
[31584.544566] Modules linked in: nfsv3 nfs_layout_flexfiles rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache loop crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel ppdev crypto_simd i2c_piix4 glue_helper cryptd i2c_core nfsd pcspkr joydev parport_pc virtio_balloon parport auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi ata_piix libata virtio_blk virtio_console virtio_net crc32c_intel serio_raw virtio_pci virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod 
[31584.548989] CPU: 0 PID: 2524 Comm: dbtest Not tainted 4.13.0-rc5 #1 
[31584.549633] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 
[31584.550215] task: ffff8cc1db4add00 task.stack: ffffba7941c10000 
[31584.550817] RIP: 0010:nfs_pageio_add_request+0x265/0x280 [nfs] 
[31584.551371] RSP: 0018:ffffba7941c13ad8 EFLAGS: 00010246 
[31584.551876] RAX: 0000000000000001 RBX: ffffba7941c13c90 RCX: 0000000000000000 
[31584.552547] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8cc1f65cfb00 
[31584.553227] RBP: ffffba7941c13b08 R08: ffffffffffffffff R09: ffffffffffffffff 
[31584.553941] R10: ffff8cc1fffd3d01 R11: 0000000000000001 R12: ffff8cc1f7821780 
[31584.554648] R13: ffff8cc1f7821780 R14: ffffba7941c13d90 R15: 0000000000000000 
[31584.555352] FS:  00007f7500d42740(0000) GS:ffff8cc1ffc00000(0000) knlGS:0000000000000000 
[31584.556126] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[31584.556705] CR2: 00007f74ff825000 CR3: 0000000134ee6000 CR4: 00000000001406f0 
[31584.557402] Call Trace: 
[31584.557652]  nfs_do_writepage+0x1af/0x3b0 [nfs] 
[31584.558103]  nfs_writepages_callback+0x14/0x30 [nfs] 
[31584.558562]  write_cache_pages+0x26f/0x510 
[31584.558951]  ? nfs_do_writepage+0x3b0/0x3b0 [nfs] 
[31584.559415]  nfs_writepages+0xc2/0x120 [nfs] 
[31584.559858]  do_writepages+0x1c/0x70 
[31584.560203]  __filemap_fdatawrite_range+0xc6/0x100 
[31584.560660]  filemap_write_and_wait_range+0x41/0xa0 
[31584.561142]  nfs_file_fsync+0x34/0x1f0 [nfs] 
[31584.561578]  vfs_fsync_range+0x3d/0xb0 
[31584.561952]  SyS_msync+0x16e/0x1f0 
[31584.562286]  do_syscall_64+0x67/0x150 
[31584.562646]  entry_SYSCALL64_slow_path+0x25/0x25 
[31584.563096] RIP: 0033:0x7f7500315dd0 
[31584.563446] RSP: 002b:00007ffe013f6b48 EFLAGS: 00000246 ORIG_RAX: 000000000000001a 
[31584.564179] RAX: ffffffffffffffda RBX: 0000000002412030 RCX: 00007f7500315dd0 
[31584.564859] RDX: 0000000000000006 RSI: 0000000000400000 RDI: 00007f74ff434000 
[31584.565541] RBP: ffffffff00000000 R08: 00007f74ff733ff0 R09: 0000000000000000 
[31584.566224] R10: 00007ffe013f6860 R11: 0000000000000246 R12: 0000000002412070 
[31584.566932] R13: 00000000000003e0 R14: 00000000000003e0 R15: 0000000000000002 
[31584.567639] Code: 1f 80 00 00 00 00 e9 57 ff ff ff 89 c2 66 0f 1f 84 00 00 00 00 00 e9 a8 fe ff ff 0f ff 66 0f 1f 84 00 00 00 00 00 e9 3c fe ff ff <0f> ff 66 0f 1f 84 00 00 00 00 00 e9 f2 fd ff ff 90 66 2e 0f 1f  
[31584.569405] ---[ end trace bd32d545f83d8d2f ]--- 
[31584.572185] BUG: unable to handle kernel paging request at 0000000400000040 
[31584.572904] IP: __nfs_pageio_add_request+0x6f/0x3c0 [nfs] 
[31584.573450] PGD 11c8d6067  
[31584.573451] P4D 11c8d6067  
[31584.573717] PUD 0  
[31584.574003]  
[31584.574369] Oops: 0000 [#1] SMP 
[31584.574702] Modules linked in: nfsv3 nfs_layout_flexfiles rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache loop crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel ppdev crypto_simd i2c_piix4 glue_helper cryptd i2c_core nfsd pcspkr joydev parport_pc virtio_balloon parport auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi ata_piix libata virtio_blk virtio_console virtio_net crc32c_intel serio_raw virtio_pci virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod 
[31584.578980] CPU: 0 PID: 2524 Comm: dbtest Tainted: G        W       4.13.0-rc5 #1 
[31584.579706] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 
[31584.580247] task: ffff8cc1db4add00 task.stack: ffffba7941c10000 
[31584.580793] RIP: 0010:__nfs_pageio_add_request+0x6f/0x3c0 [nfs] 
[31584.581314] RSP: 0018:ffffba7941c13a88 EFLAGS: 00010286 
[31584.581809] RAX: ffff8cc1f77b3080 RBX: ffffba7941c13c90 RCX: 0000000000000000 
[31584.582519] RDX: 0000000400000000 RSI: ffffdd3b848f3600 RDI: ffffba7941c13c90 
[31584.583218] RBP: ffffba7941c13ac8 R08: ffffffffffffffff R09: ffffffffffffffff 
[31584.583882] R10: ffff8cc1fffd3d01 R11: 0000000000000001 R12: 0000000000001000 
[31584.584576] R13: ffff8cc1f7821780 R14: ffffba7941c13cf0 R15: ffff8cc1f7821780 
[31584.585243] FS:  00007f7500d42740(0000) GS:ffff8cc1ffc00000(0000) knlGS:0000000000000000 
[31584.586045] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[31584.586614] CR2: 00007f1c908a5000 CR3: 0000000134ee6000 CR4: 00000000001406f0 
[31584.587309] Call Trace: 
[31584.587581]  nfs_pageio_add_request+0xac/0x280 [nfs] 
[31584.588073]  nfs_do_writepage+0x1af/0x3b0 [nfs] 
[31584.588546]  nfs_writepages_callback+0x14/0x30 [nfs] 
[31584.589043]  write_cache_pages+0x26f/0x510 
[31584.589432]  ? nfs_do_writepage+0x3b0/0x3b0 [nfs] 
[31584.589904]  nfs_writepages+0xc2/0x120 [nfs] 
[31584.590340]  do_writepages+0x1c/0x70 
[31584.590718]  __filemap_fdatawrite_range+0xc6/0x100 
[31584.591184]  filemap_write_and_wait_range+0x41/0xa0 
[31584.591657]  nfs_file_fsync+0x34/0x1f0 [nfs] 
[31584.592076]  vfs_fsync_range+0x3d/0xb0 
[31584.592446]  SyS_msync+0x16e/0x1f0 
[31584.592795]  do_syscall_64+0x67/0x150 
[31584.593177]  entry_SYSCALL64_slow_path+0x25/0x25 
[31584.593640] RIP: 0033:0x7f7500315dd0 
[31584.594028] RSP: 002b:00007ffe013f6b48 EFLAGS: 00000246 ORIG_RAX: 000000000000001a 
[31584.594752] RAX: ffffffffffffffda RBX: 0000000002412030 RCX: 00007f7500315dd0 
[31584.595452] RDX: 0000000000000006 RSI: 0000000000400000 RDI: 00007f74ff434000 
[31584.596166] RBP: ffffffff00000000 R08: 00007f74ff733ff0 R09: 0000000000000000 
[31584.596900] R10: 00007ffe013f6860 R11: 0000000000000246 R12: 0000000002412070 
[31584.597611] R13: 00000000000003e0 R14: 00000000000003e0 R15: 0000000000000002 
[31584.598284] Code: 48 89 df e8 34 f4 ff ff 48 83 78 18 00 49 89 c6 0f 84 ae 00 00 00 48 8b 70 08 48 85 f6 0f 84 ca 00 00 00 48 8b 56 18 49 8b 47 18 <48> 8b 4a 40 48 39 48 40 0f 84 a6 01 00 00 41 8b 57 38 41 8b 47  
[31584.600128] RIP: __nfs_pageio_add_request+0x6f/0x3c0 [nfs] RSP: ffffba7941c13a88 
[31584.600817] CR2: 0000000400000040 
[31584.601200] ---[ end trace bd32d545f83d8d30 ]--- 
[31584.601667] Kernel panic - not syncing: Fatal exception 
[31584.602973] Kernel Offset: 0x37200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) 
[31584.604011] ---[ end Kernel panic - not syncing: Fatal exception 
[-- MARK -- Fri Aug 18 17:25:00 2017] 
[-- MARK -- Fri Aug 18 17:30:00 2017]
Comment 1 JianhongYin 2017-08-19 12:32:53 UTC 
reproduced on v4.12 and test pass on v4.11

so this is a regression of v4.12
Comment 2 Trond Myklebust 2017-08-19 19:05:52 UTC 
Created attachment 258025 [details]
NFSv4: Fix up mirror allocation

Hi Jianhong

Can you see if the proposed patch fixes the issue for you? As far as I can tell, it is not a regression, but a longstanding bug, so I'd like to make it a stable patch.

Thanks
  Trond
Comment 3 JianhongYin 2017-08-20 03:10:24 UTC 
(In reply to Trond Myklebust from comment #2)
> Created attachment 258025 [details]
> NFSv4: Fix up mirror allocation
> 
> Hi Jianhong
> 
> Can you see if the proposed patch fixes the issue for you? As far as I can
> tell, it is not a regression, but a longstanding bug, so I'd like to make it
> a stable patch.
Hi Trond

Your patch works!

'''
[23:05:54 root@ /var/lib/xfstests]# DIFF_LENGTH=-0 ./check -nfs $fstest
FSTYP         -- nfs
PLATFORM      -- Linux/x86_64 bkr-hv03-guest34 4.13.0-rc6
MKFS_OPTIONS  -- 10.66.12.250:/nfs_pub2
MOUNT_OPTIONS -- -overs=4.1 -o context=system_u:object_r:root_t:s0 10.66.12.250:/nfs_pub2 /mnt/nfsmp2-xfstests-sh-20959

generic/010      29s
Ran: generic/010
Passed all 1 tests

:: [   PASS   ] :: Running 'DIFF_LENGTH=-0 ./check -nfs $fstest' (Expected 0, got 0)
[root@bkr-hv03-guest34 single-host]# uname -r
4.13.0-rc6    #Comment: I rename the patched kernel to rc6
'''

> 
> Thanks
>   Trond