Bug 216151

Summary: kernel panic after BUG: KASAN: use-after-free in _copy_to_iter+0x830/0x1030
Product: File System Reporter: Zorro Lang (zlang)
Component: XFSAssignee: FileSystem/XFS Default Virtual Assignee (filesystem_xfs)
Status: NEW ---    
Severity: normal CC: chuck.lever
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: v5.19-rc2+ Tree: Mainline
Regression: No

Description Zorro Lang 2022-06-20 05:52:15 UTC
xfstests generic/465 hit below kernel panic and KASAN BUG on NFS through XFS(default mkfs options). Hit on linux v5.19-rc2+, which HEAD is:

commit 05c6ca8512f2722f57743d653bb68cf2a273a55a
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Sun Jun 19 09:58:28 2022 -0500

    Merge tag 'x86-urgent-2022-06-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip


# cat local.config
FSTYP=nfs
TEST_DEV=$mynfs_server:/mnt/xfstests/test/nfs-server
TEST_DIR=/mnt/xfstests/test/nfs-client
SCRATCH_DEV=$mynfs_server:/mnt/xfstests/scratch/nfs-server
SCRATCH_MNT=/mnt/xfstests/scratch/nfs-client
MOUNT_OPTIONS="-o vers=4.2"
TEST_FS_MOUNT_OPTS="-o vers=4.2"

XFS info:
meta-data=/dev/vda4              isize=512    agcount=4, agsize=983040 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1    bigtime=1 inobtcount=1
data     =                       bsize=4096   blocks=3932160, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=16384, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

console log:
[26844.323108] run fstests generic/465 at 2022-06-20 00:24:32 
[26847.872804] ================================================================== 
[26847.872854] BUG: KASAN: use-after-free in _copy_to_iter+0x694/0xd0c 
[26847.872992] Write of size 16 at addr ffff2fb1d4013000 by task nfsd/45920 
[26847.872999]  
[26847.873083] CPU: 0 PID: 45920 Comm: nfsd Kdump: loaded Not tainted 5.19.0-rc2+ #1 
[26847.873090] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015 
[26847.873094] Call trace: 
[26847.873174]  dump_backtrace+0x1e0/0x26c 
[26847.873198]  show_stack+0x1c/0x70 
[26847.873203]  dump_stack_lvl+0x98/0xd0 
[26847.873262]  print_address_description.constprop.0+0x74/0x420 
[26847.873285]  print_report+0xc8/0x234 
[26847.873290]  kasan_report+0xb0/0xf0 
[26847.873294]  kasan_check_range+0xf4/0x1a0 
[26847.873298]  memcpy+0xdc/0x100 
[26847.873303]  _copy_to_iter+0x694/0xd0c 
[26847.873307]  copy_page_to_iter+0x3f0/0xb30 
[26847.873311]  filemap_read+0x3e8/0x7e0 
[26847.873319]  generic_file_read_iter+0x2b0/0x404 
[26847.873324]  xfs_file_buffered_read+0x18c/0x4e0 [xfs] 
[26847.873854]  xfs_file_read_iter+0x260/0x514 [xfs] 
[26847.874168]  do_iter_readv_writev+0x338/0x4b0 
[26847.874176]  do_iter_read+0x120/0x374 
[26847.874180]  vfs_iter_read+0x5c/0xa0 
[26847.874185]  nfsd_readv+0x1a0/0x9ac [nfsd] 
[26847.874308]  nfsd4_encode_read_plus_data+0x2f0/0x690 [nfsd] 
[26847.874387]  nfsd4_encode_read_plus+0x344/0x924 [nfsd] 
[26847.874468]  nfsd4_encode_operation+0x1fc/0x800 [nfsd] 
[26847.874544]  nfsd4_proc_compound+0x9c4/0x2364 [nfsd] 
[26847.874620]  nfsd_dispatch+0x3a4/0x67c [nfsd] 
[26847.874697]  svc_process_common+0xd54/0x1be0 [sunrpc] 
[26847.874921]  svc_process+0x298/0x484 [sunrpc] 
[26847.875063]  nfsd+0x2b0/0x580 [nfsd] 
[26847.875143]  kthread+0x230/0x294 
[26847.875170]  ret_from_fork+0x10/0x20 
[26847.875178]  
[26847.875180] Allocated by task 602477: 
[26847.875185]  kasan_save_stack+0x28/0x50 
[26847.875191]  __kasan_slab_alloc+0x68/0x90 
[26847.875195]  kmem_cache_alloc+0x180/0x394 
[26847.875199]  security_inode_alloc+0x30/0x120 
[26847.875221]  inode_init_always+0x49c/0xb1c 
[26847.875228]  alloc_inode+0x70/0x1c0 
[26847.875232]  new_inode+0x20/0x230 
[26847.875236]  debugfs_create_dir+0x74/0x48c 
[26847.875243]  rpc_clnt_debugfs_register+0xd0/0x174 [sunrpc] 
[26847.875384]  rpc_client_register+0x90/0x4c4 [sunrpc] 
[26847.875526]  rpc_new_client+0x6e0/0x1260 [sunrpc] 
[26847.875666]  __rpc_clone_client+0x158/0x7d4 [sunrpc] 
[26847.875831]  rpc_clone_client+0x168/0x1dc [sunrpc] 
[26847.875972]  nfs4_proc_lookup_mountpoint+0x180/0x1f0 [nfsv4] 
[26847.876149]  nfs4_submount+0xcc/0x6cc [nfsv4] 
[26847.876251]  nfs_d_automount+0x4b4/0x7bc [nfs] 
[26847.876389]  __traverse_mounts+0x180/0x4a0 
[26847.876396]  step_into+0x510/0x940 
[26847.876400]  walk_component+0xf0/0x510 
[26847.876405]  link_path_walk.part.0.constprop.0+0x4c0/0xa3c 
[26847.876410]  path_lookupat+0x6c/0x57c 
[26847.876436]  filename_lookup+0x13c/0x400 
[26847.876440]  vfs_path_lookup+0xa0/0xec 
[26847.876445]  mount_subtree+0x1c4/0x380 
[26847.876451]  do_nfs4_mount+0x3c0/0x770 [nfsv4] 
[26847.876554]  nfs4_try_get_tree+0xc0/0x24c [nfsv4] 
[26847.876653]  nfs_get_tree+0xc0/0x110 [nfs] 
[26847.876742]  vfs_get_tree+0x78/0x2a0 
[26847.876748]  do_new_mount+0x228/0x4fc 
[26847.876753]  path_mount+0x268/0x16d4 
[26847.876757]  __arm64_sys_mount+0x1dc/0x240 
[26847.876762]  invoke_syscall.constprop.0+0xd8/0x1d0 
[26847.876769]  el0_svc_common.constprop.0+0x224/0x2bc 
[26847.876774]  do_el0_svc+0x4c/0x90 
[26847.876778]  el0_svc+0x5c/0x140 
[26847.876785]  el0t_64_sync_handler+0xb4/0x130 
[26847.876789]  el0t_64_sync+0x174/0x178 
[26847.876793]  
[26847.876794] Last potentially related work creation: 
[26847.876797]  kasan_save_stack+0x28/0x50 
[26847.876802]  __kasan_record_aux_stack+0x9c/0xc0 
[26847.876806]  kasan_record_aux_stack_noalloc+0x10/0x20 
[26847.876811]  call_rcu+0xf8/0x6c0 
[26847.876818]  security_inode_free+0x94/0xc0 
[26847.876823]  __destroy_inode+0xb0/0x420 
[26847.876828]  destroy_inode+0x80/0x170 
[26847.876832]  evict+0x334/0x4c0 
[26847.876836]  iput_final+0x138/0x364 
[26847.876841]  iput.part.0+0x330/0x47c 
[26847.876845]  iput+0x44/0x60 
[26847.876849]  dentry_unlink_inode+0x200/0x43c 
[26847.876853]  __dentry_kill+0x29c/0x56c 
[26847.876857]  dput+0x41c/0x870 
[26847.876860]  simple_recursive_removal+0x4ac/0x630 
[26847.876865]  debugfs_remove+0x5c/0x80 
[26847.876870]  rpc_clnt_debugfs_unregister+0x3c/0x7c [sunrpc] 
[26847.877011]  rpc_free_client_work+0xdc/0x480 [sunrpc] 
[26847.877154]  process_one_work+0x794/0x184c 
[26847.877161]  worker_thread+0x3d4/0xc40 
[26847.877165]  kthread+0x230/0x294 
[26847.877168]  ret_from_fork+0x10/0x20 
[26847.877172]  
[26847.877174] Second to last potentially related work creation: 
[26847.877177]  kasan_save_stack+0x28/0x50 
[26847.877181]  __kasan_record_aux_stack+0x9c/0xc0 
[26847.877185]  kasan_record_aux_stack_noalloc+0x10/0x20 
[26847.877190]  call_rcu+0xf8/0x6c0 
[26847.877195]  security_inode_free+0x94/0xc0 
[26847.877200]  __destroy_inode+0xb0/0x420 
[26847.877205]  destroy_inode+0x80/0x170 
[26847.877209]  evict+0x334/0x4c0 
[26847.877213]  iput_final+0x138/0x364 
[26847.877217]  iput.part.0+0x330/0x47c 
[26847.877221]  iput+0x44/0x60 
[26847.877226]  dentry_unlink_inode+0x200/0x43c 
[26847.877229]  __dentry_kill+0x29c/0x56c 
[26847.877233]  dput+0x44c/0x870 
[26847.877237]  __fput+0x244/0x730 
[26847.877241]  ____fput+0x14/0x20 
[26847.877245]  task_work_run+0xd0/0x240 
[26847.877250]  do_exit+0x3a0/0xaac 
[26847.877256]  do_group_exit+0xac/0x244 
[26847.877260]  __arm64_sys_exit_group+0x40/0x4c 
[26847.877264]  invoke_syscall.constprop.0+0xd8/0x1d0 
[26847.877270]  el0_svc_common.constprop.0+0x224/0x2bc 
[26847.877275]  do_el0_svc+0x4c/0x90 
[26847.877280]  el0_svc+0x5c/0x140 
[26847.877284]  el0t_64_sync_handler+0xb4/0x130 
[26847.877288]  el0t_64_sync+0x174/0x178 
[26847.877292]  
[26847.877293] The buggy address belongs to the object at ffff2fb1d4013000 
[26847.877293]  which belongs to the cache lsm_inode_cache of size 128 
[26847.877298] The buggy address is located 0 bytes inside of 
[26847.877298]  128-byte region [ffff2fb1d4013000, ffff2fb1d4013080) 
[26847.877302]  
[26847.877304] The buggy address belongs to the physical page: 
[26847.877308] page:000000007bc4a504 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff2fb1d4013000 pfn:0x154013 
[26847.877363] flags: 0x17ffff800000200(slab|node=0|zone=2|lastcpupid=0xfffff) 
[26847.877375] raw: 017ffff800000200 fffffcbec6646688 fffffcbec750d708 ffff2fb1808dfe00 
[26847.877379] raw: ffff2fb1d4013000 0000000000150010 00000001ffffffff 0000000000000000 
[26847.877382] page dumped because: kasan: bad access detected 
[26847.877384]  
[26847.877385] Memory state around the buggy address: 
[26847.877389]  ffff2fb1d4012f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
[26847.877392]  ffff2fb1d4012f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
[26847.877395] >ffff2fb1d4013000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb 
[26847.877397]                    ^ 
[26847.877400]  ffff2fb1d4013080: fc fc fc fc fc fc fc fc fa fb fb fb fb fb fb fb 
[26847.877402]  ffff2fb1d4013100: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc 
[26847.877405] ================================================================== 
[26847.877570] Disabling lock debugging due to kernel taint 
[26848.391268] Unable to handle kernel write to read-only memory at virtual address ffff2fb197f76000 
[26848.393628] KASAN: maybe wild-memory-access in range [0xfffd7d8cbfbb0000-0xfffd7d8cbfbb0007] 
[26848.395572] Mem abort info: 
[26848.396408]   ESR = 0x000000009600004f 
[26848.397314]   EC = 0x25: DABT (current EL), IL = 32 bits 
[26848.398520]   SET = 0, FnV = 0 
[26848.506889]   EA = 0, S1PTW = 0 
[26848.507633]   FSC = 0x0f: level 3 permission fault 
[26848.508802] Data abort info: 
[26848.509480]   ISV = 0, ISS = 0x0000004f 
[26848.510347]   CM = 0, WnR = 1 
[26848.511032] swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000b22dd000 
[26848.512543] [ffff2fb197f76000] pgd=18000001bfff8003, p4d=18000001bfff8003, pud=18000001bfa08003, pmd=18000001bf948003, pte=0060000117f76f87 
[26848.515600] Internal error: Oops: 9600004f [#1] SMP 
[26848.516870] Modules linked in: loop dm_mod tls rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace rfkill sunrpc vfat fat drm fuse xfs libcrc32c crct10dif_ce ghash_ce sha2_ce sha256_arm64 sha1_ce virtio_blk virtio_net virtio_console net_failover failover virtio_mmio ipmi_devintf ipmi_msghandler 
[26848.525472] CPU: 1 PID: 45919 Comm: nfsd Kdump: loaded Tainted: G    B             5.19.0-rc2+ #1 
[26848.527934] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015 
[26848.529819] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) 
[26848.531625] pc : __memcpy+0x2c/0x230 
[26848.532583] lr : memcpy+0xa8/0x100 
[26848.533497] sp : ffff80000bbb6f00 
[26848.534444] x29: ffff80000bbb6f00 x28: 0000000000000000 x27: ffff2fb18a4bd5b8 
[26848.536435] x26: 0000000000000000 x25: ffff80000bbb7740 x24: ffff2fb18a4bd5b0 
[26848.538283] x23: ffff2fb1ee80bff0 x22: ffffa83e4692e000 x21: ffffa83e434ae3e8 
[26848.540181] x20: ffff2fb197f76000 x19: 0000000000000010 x18: ffff2fb1d3c34530 
[26848.542071] x17: 0000000000000000 x16: ffffa83e42d01a30 x15: 6161616161616161 
[26848.543840] x14: 6161616161616161 x13: 6161616161616161 x12: 6161616161616161 
[26848.545614] x11: 1fffe5f632feec01 x10: ffff65f632feec01 x9 : dfff800000000000 
[26848.547387] x8 : ffff2fb197f7600f x7 : 6161616161616161 x6 : 6161616161616161 
[26848.549156] x5 : ffff2fb197f76010 x4 : ffff2fb1ee80c000 x3 : ffffa83e434ae3e8 
[26848.550924] x2 : 0000000000000010 x1 : ffff2fb1ee80bff0 x0 : ffff2fb197f76000 
[26848.552694] Call trace: 
[26848.553314]  __memcpy+0x2c/0x230 
[26848.554123]  _copy_to_iter+0x694/0xd0c 
[26848.555084]  copy_page_to_iter+0x3f0/0xb30 
[26848.556104]  filemap_read+0x3e8/0x7e0 
[26848.557020]  generic_file_read_iter+0x2b0/0x404 
[26848.558152]  xfs_file_buffered_read+0x18c/0x4e0 [xfs] 
[26848.559795]  xfs_file_read_iter+0x260/0x514 [xfs] 
[26848.561265]  do_iter_readv_writev+0x338/0x4b0 
[26848.562346]  do_iter_read+0x120/0x374 
[26848.563263]  vfs_iter_read+0x5c/0xa0 
[26848.564162]  nfsd_readv+0x1a0/0x9ac [nfsd] 
[26848.565415]  nfsd4_encode_read_plus_data+0x2f0/0x690 [nfsd] 
[26848.566869]  nfsd4_encode_read_plus+0x344/0x924 [nfsd] 
[26848.568231]  nfsd4_encode_operation+0x1fc/0x800 [nfsd] 
[26848.569596]  nfsd4_proc_compound+0x9c4/0x2364 [nfsd] 
[26848.570908]  nfsd_dispatch+0x3a4/0x67c [nfsd] 
[26848.572067]  svc_process_common+0xd54/0x1be0 [sunrpc] 
[26848.573508]  svc_process+0x298/0x484 [sunrpc] 
[26848.574743]  nfsd+0x2b0/0x580 [nfsd] 
[26848.575718]  kthread+0x230/0x294 
[26848.576528]  ret_from_fork+0x10/0x20 
[26848.577421] Code: f100405f 540000c3 a9401c26 a97f348c (a9001c06)  
[26848.578934] SMP: stopping secondary CPUs 
[26848.582664] Starting crashdump kernel... 
[26848.583602] Bye!
Comment 1 Zorro Lang 2022-06-20 06:07:49 UTC
# ./scripts/decode_stacktrace.sh vmlinux < crash.log

[26844.323108] run fstests generic/465 at 2022-06-20 00:24:32                                                                                                                                  [26847.872804] ==================================================================                                                                                                              
[26847.872854] BUG: KASAN: use-after-free in _copy_to_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/iov_iter.c:667 (discriminator 31)) 
[26847.872992] Write of size 16 at addr ffff2fb1d4013000 by task nfsd/45920                                                                                                                    
[26847.872999]                                                                                                                                                                                 
[26847.873090] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015                                                                                                                  
[26847.873094] Call trace:                                                                                                                                                                     [26847.873174] dump_backtrace (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/stacktrace.c:200)                                                               
[26847.873198] show_stack (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/stacktrace.c:207)                                                                   
[26847.873203] dump_stack_lvl (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/dump_stack.c:107 (discriminator 4))                                                           
[26847.873262] print_address_description.constprop.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/mm.h:848 /mnt/tests/kernel/distribution/upstream-kernel/ins
tall/kernel/mm/kasan/report.c:210 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/report.c:311)                                                                         
[26847.873285] print_report (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/report.c:390 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/report.
c:430)                                                                                                                                                                                         
[26847.873290] kasan_report (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/report.c:162 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/report.
c:493)                                                                                                                                                                                         
[26847.873294] kasan_check_range (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/generic.c:173 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/g
eneric.c:189)                                                                                                                                                                                  
[26847.873298] memcpy (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/shadow.c:65 (discriminator 1))  
[26847.873303] _copy_to_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/iov_iter.c:667 (discriminator 31))                                                             
[26847.873307] copy_page_to_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/iov_iter.c:855 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/iov_iter.c
:880)                                                                                                                                                                                          
[26847.873311] filemap_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/uio.h:153 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/filemap.c
:2730)                                                                                                                                                                                         
[26847.873319] generic_file_read_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/filemap.c:2825)                                                                        
[26847.873324] xfs_file_buffered_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/xfs/xfs_file.c:270) xfs
[26847.873854] xfs_file_read_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/xfs/xfs_file.c:295) xfs               
[26847.874168] do_iter_readv_writev (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/fs.h:2052 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/r
ead_write.c:740)                                                                               
[26847.874176] do_iter_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/read_write.c:803)         
[26847.874180] vfs_iter_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/read_write.c:846)
[26847.874185] nfsd_readv (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/vfs.c:931) nfsd                                                                     [175/1812]
[26847.874308] nfsd4_encode_read_plus_data (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4xdr.c:4762) nfsd
[26847.874387] nfsd4_encode_read_plus (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4xdr.c:4795 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nf
sd/nfs4xdr.c:4854) nfsd                                                                                                                                                                        
[26847.874468] nfsd4_encode_operation (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4xdr.c:5323 (discriminator 4)) nfsd                                            
[26847.874544] nfsd4_proc_compound (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4proc.c:2757) nfsd
[26847.874620] nfsd_dispatch (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfssvc.c:1056) nfsd
[26847.874697] svc_process_common (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/svc.c:1339) sunrpc
[26847.874921] svc_process (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/svc.c:1470) sunrpc
[26847.875063] nfsd (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfssvc.c:979) nfsd
[26847.875143] kthread (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/kthread.c:376) 
[26847.875170] ret_from_fork (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry.S:868) 
[26847.875178]
[26847.875180] Allocated by task 602477:
[26847.875185] kasan_save_stack (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/common.c:39) 
[26847.875191] __kasan_slab_alloc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/common.c:45 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/co
mmon.c:436 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/common.c:469)  
[26847.875195] kmem_cache_alloc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/slab.h:750 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/slub.c:3214 /mnt/
tests/kernel/distribution/upstream-kernel/install/kernel/mm/slub.c:3222 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/slub.c:3229 /mnt/tests/kernel/distribution/upstream-ke
rnel/install/kernel/mm/slub.c:3239) 
[26847.875199] security_inode_alloc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/security/security.c:594 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/securi
ty/security.c:1024) 
[26847.875221] inode_init_always (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:195) 
[26847.875228] alloc_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:267) 
[26847.875232] new_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1018 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1047) 
[26847.875236] debugfs_create_dir (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/debugfs/inode.c:72 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/debugfs
/inode.c:578) 
[26847.875243] rpc_clnt_debugfs_register (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/debugfs.c:157) sunrpc
[26847.875384] rpc_client_register (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/clnt.c:306) sunrpc
[26847.875526] rpc_new_client (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/clnt.c:431) sunrpc
[26847.875666] __rpc_clone_client (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/clnt.c:642) sunrpc
[26847.875831] rpc_clone_client (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/clnt.c:670) sunrpc
[26847.875972] nfs4_proc_lookup_mountpoint (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfs/nfs4proc.c:4507 (discriminator 1)) nfsv4
[26847.876149] nfs4_submount (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfs/nfs4namespace.c:460) nfsv4
[26847.876251] nfs_d_automount (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfs/namespace.c:189) nfs
[26847.876389] __traverse_mounts (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:1355 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:1400) 
[26847.876396] step_into (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:1539 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:1844) 
[26847.876400] walk_component (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:2020) 
[26847.876405] link_path_walk.part.0.constprop.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:2341) 
[26847.876410] path_lookupat (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:2466 (discriminator 2) /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/
namei.c:2492 (discriminator 2)) 
[26847.876436] filename_lookup (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:2522) 
[26847.876440] vfs_path_lookup (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namei.c:2638) 
[26847.876445] mount_subtree (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namespace.c:3549)
[26847.876451] do_nfs4_mount (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfs/nfs4super.c:206) nfsv4                                                                      
[26847.876554] nfs4_try_get_tree (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfs/nfs4super.c:226 (discriminator 3)) nfsv4
[26847.876653] nfs_get_tree (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfs/fs_context.c:1433) nfs
[26847.876742] vfs_get_tree (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/super.c:1497) 
[26847.876748] do_new_mount (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namespace.c:3040) 
[26847.876753] path_mount (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namespace.c:3370) 
[26847.876757] __arm64_sys_mount (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namespace.c:3383 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namespace.
c:3591 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namespace.c:3568 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/namespace.c:3568) 
[26847.876762] invoke_syscall.constprop.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/syscall.c:38 /mnt/tests/kernel/distribution/upstream-kernel/install/
kernel/arch/arm64/kernel/syscall.c:52) 
[26847.876769] el0_svc_common.constprop.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/syscall.c:158) 
[26847.876774] do_el0_svc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/syscall.c:207) 
[26847.876778] el0_svc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry-common.c:133 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/a
rm64/kernel/entry-common.c:142 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry-common.c:625) 
[26847.876785] el0t_64_sync_handler (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry-common.c:643) 
[26847.876789] el0t_64_sync (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry.S:581) 
[26847.876793]
[26847.876794] Last potentially related work creation:
[26847.876797] kasan_save_stack (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/common.c:39) 
[26847.876802] __kasan_record_aux_stack (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/generic.c:348) 
[26847.876806] kasan_record_aux_stack_noalloc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/generic.c:359) 
[26847.876811] call_rcu (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/rcu/tree.c:3127) 
[26847.876818] security_inode_free (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/security/security.c:1058) 
[26847.876823] __destroy_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/fsnotify.h:176 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/i
node.c:286) 
[26847.876828] destroy_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:309 (discriminator 2)) 
[26847.876832] evict (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:680 (discriminator 2)) 
[26847.876836] iput_final (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1745) 
[26847.876841] iput.part.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1772) 
[26847.876845] iput (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1772 (discriminator 2)) 
[26847.876849] dentry_unlink_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/dcache.c:402) 
[26847.876853] __dentry_kill (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./arch/arm64/include/asm/current.h:19 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel
/./arch/arm64/include/asm/preempt.h:47 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/dcache.c:610) 
[26847.876857] dput (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/dcache.c:896) 
[26847.876860] simple_recursive_removal (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/libfs.c:312) 
[26847.876865] debugfs_remove (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/debugfs/inode.c:743 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/debugfs/in
ode.c:736) 
[26847.876870] rpc_clnt_debugfs_unregister (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/debugfs.c:170) sunrpc
[26847.877011] rpc_free_client_work (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/clnt.c:357 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunr
pc/clnt.c:897) sunrpc
[26847.877154] process_one_work (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/workqueue.c:2294) 
[26847.877161] worker_thread (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/list.h:292 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/wor
kqueue.c:2437)

[26847.877165] kthread (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/kthread.c:376)                                                                           [88/1812]
[26847.877168] ret_from_fork (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry.S:868) 
[26847.877172]
[26847.877174] Second to last potentially related work creation:
[26847.877177] kasan_save_stack (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/common.c:39) 
[26847.877181] __kasan_record_aux_stack (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/generic.c:348) 
[26847.877185] kasan_record_aux_stack_noalloc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/generic.c:359) 
[26847.877190] call_rcu (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/rcu/tree.c:3127) 
[26847.877195] security_inode_free (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/security/security.c:1058) 
[26847.877200] __destroy_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/fsnotify.h:176 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/i
node.c:286) 
[26847.877205] destroy_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:309 (discriminator 2)) 
[26847.877209] evict (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:680 (discriminator 2)) 
[26847.877213] iput_final (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1745) 
[26847.877217] iput.part.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1772) 
[26847.877221] iput (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/inode.c:1772 (discriminator 2)) 
[26847.877226] dentry_unlink_inode (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/dcache.c:402) 
[26847.877229] __dentry_kill (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./arch/arm64/include/asm/current.h:19 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel
/./arch/arm64/include/asm/preempt.h:47 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/dcache.c:610) 
[26847.877233] dput (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/dcache.c:896) 
[26847.877237] __fput (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/file_table.c:331) 
[26847.877241] ____fput (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/file_table.c:351) 
[26847.877245] task_work_run (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/task_work.c:179 (discriminator 1)) 
[26847.877250] do_exit (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/exit.c:804) 
[26847.877256] do_group_exit (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/exit.c:906) 
[26847.877260] __arm64_sys_exit_group (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/exit.c:934) 
[26847.877264] invoke_syscall.constprop.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/syscall.c:38 /mnt/tests/kernel/distribution/upstream-kernel/install/
kernel/arch/arm64/kernel/syscall.c:52) 
[26847.877270] el0_svc_common.constprop.0 (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/syscall.c:158) 
[26847.877275] do_el0_svc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/syscall.c:207) 
[26847.877280] el0_svc (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry-common.c:133 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/a
rm64/kernel/entry-common.c:142 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry-common.c:625) 
[26847.877284] el0t_64_sync_handler (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry-common.c:643) 
[26847.877288] el0t_64_sync (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry.S:581) 
[26847.877292]
[26847.877293] The buggy address belongs to the object at ffff2fb1d4013000
[26847.877293]  which belongs to the cache lsm_inode_cache of size 128
[26847.877298] The buggy address is located 0 bytes inside of
[26847.877298]  128-byte region [ffff2fb1d4013000, ffff2fb1d4013080)
[26847.877302]
[26847.877304] The buggy address belongs to the physical page:
[26847.877308] page:000000007bc4a504 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff2fb1d4013000 pfn:0x154013                                                             [47/1812]
[26847.877363] flags: 0x17ffff800000200(slab|node=0|zone=2|lastcpupid=0xfffff)
[26847.877375] raw: 017ffff800000200 fffffcbec6646688 fffffcbec750d708 ffff2fb1808dfe00
[26847.877379] raw: ffff2fb1d4013000 0000000000150010 00000001ffffffff 0000000000000000
[26847.877382] page dumped because: kasan: bad access detected
[26847.877384]
[26847.877385] Memory state around the buggy address:
[26847.877389]  ffff2fb1d4012f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[26847.877392]  ffff2fb1d4012f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[26847.877395] >ffff2fb1d4013000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[26847.877397]                    ^
[26847.877400]  ffff2fb1d4013080: fc fc fc fc fc fc fc fc fa fb fb fb fb fb fb fb
[26847.877402]  ffff2fb1d4013100: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[26847.877405] ==================================================================
[26847.877570] Disabling lock debugging due to kernel taint
[26848.391268] Unable to handle kernel write to read-only memory at virtual address ffff2fb197f76000
[26848.393628] KASAN: maybe wild-memory-access in range [0xfffd7d8cbfbb0000-0xfffd7d8cbfbb0007] 
[26848.395572] Mem abort info:
[26848.396408]   ESR = 0x000000009600004f
[26848.397314]   EC = 0x25: DABT (current EL), IL = 32 bits
[26848.398520]   SET = 0, FnV = 0
[26848.506889]   EA = 0, S1PTW = 0
[26848.507633]   FSC = 0x0f: level 3 permission fault
[26848.508802] Data abort info:
[26848.509480]   ISV = 0, ISS = 0x0000004f
[26848.510347]   CM = 0, WnR = 1
[26848.511032] swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000b22dd000
[26848.512543] [ffff2fb197f76000] pgd=18000001bfff8003, p4d=18000001bfff8003, pud=18000001bfa08003, pmd=18000001bf948003, pte=0060000117f76f87
[26848.515600] Internal error: Oops: 9600004f [#1] SMP
[26848.516870] Modules linked in: loop dm_mod tls rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace rfkill sunrpc v
fat fat drm fuse xfs libcrc32c crct10dif_ce ghash_ce sha2_ce sha256_arm64 sha1_ce virtio_blk virtio_net virtio_console net_failover failover virtio_mmio ipmi_devintf ipmi_msghandler
[26848.527934] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[26848.529819] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[26848.531625] pc : __memcpy (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/lib/memcpy.S:73) 
[26848.532583] lr : memcpy (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/kasan/shadow.c:70) 
[26848.533497] sp : ffff80000bbb6f00
[26848.534444] x29: ffff80000bbb6f00 x28: 0000000000000000 x27: ffff2fb18a4bd5b8
[26848.536435] x26: 0000000000000000 x25: ffff80000bbb7740 x24: ffff2fb18a4bd5b0
[26848.538283] x23: ffff2fb1ee80bff0 x22: ffffa83e4692e000 x21: ffffa83e434ae3e8
[26848.540181] x20: ffff2fb197f76000 x19: 0000000000000010 x18: ffff2fb1d3c34530
[26848.542071] x17: 0000000000000000 x16: ffffa83e42d01a30 x15: 6161616161616161
[26848.543840] x14: 6161616161616161 x13: 6161616161616161 x12: 6161616161616161
[26848.545614] x11: 1fffe5f632feec01 x10: ffff65f632feec01 x9 : dfff800000000000
[26848.547387] x8 : ffff2fb197f7600f x7 : 6161616161616161 x6 : 6161616161616161
[26848.549156] x5 : ffff2fb197f76010 x4 : ffff2fb1ee80c000 x3 : ffffa83e434ae3e8
[26848.550924] x2 : 0000000000000010 x1 : ffff2fb1ee80bff0 x0 : ffff2fb197f76000
[26848.552694] Call trace:
[26848.553314] __memcpy (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/lib/memcpy.S:73) 
[26848.554123] _copy_to_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/iov_iter.c:667 (discriminator 31)) 
[26848.555084] copy_page_to_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/iov_iter.c:855 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/lib/iov_iter.c
:880) 
[26848.556104] filemap_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/uio.h:153 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/filemap.c
:2730) 
[26848.557020] generic_file_read_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/mm/filemap.c:2825) 
[26848.558152] xfs_file_buffered_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/xfs/xfs_file.c:270) xfs
[26848.559795] xfs_file_read_iter (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/xfs/xfs_file.c:295) xfs
[26848.561265] do_iter_readv_writev (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/./include/linux/fs.h:2052 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/r
ead_write.c:740) 
[26848.562346] do_iter_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/read_write.c:803) 
[26848.563263] vfs_iter_read (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/read_write.c:846) 
[26848.564162] nfsd_readv (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/vfs.c:931) nfsd
[26848.565415] nfsd4_encode_read_plus_data (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4xdr.c:4762) nfsd
[26848.566869] nfsd4_encode_read_plus (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4xdr.c:4795 /mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nf
sd/nfs4xdr.c:4854) nfsd
[26848.568231] nfsd4_encode_operation (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4xdr.c:5323 (discriminator 4)) nfsd
[26848.569596] nfsd4_proc_compound (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfs4proc.c:2757) nfsd
[26848.570908] nfsd_dispatch (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfssvc.c:1056) nfsd
[26848.572067] svc_process_common (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/svc.c:1339) sunrpc
[26848.573508] svc_process (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/net/sunrpc/svc.c:1470) sunrpc
[26848.574743] nfsd (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/fs/nfsd/nfssvc.c:979) nfsd
[26848.575718] kthread (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/kernel/kthread.c:376) 
[26848.576528] ret_from_fork (/mnt/tests/kernel/distribution/upstream-kernel/install/kernel/arch/arm64/kernel/entry.S:868) 
[26848.577421] Code: f100405f 540000c3 a9401c26 a97f348c (a9001c06)
All code
========
   0:   f100405f        cmp     x2, #0x10
   4:   540000c3        b.cc    0x1c  // b.lo, b.ul, b.last
   8:   a9401c26        ldp     x6, x7, [x1]
   c:   a97f348c        ldp     x12, x13, [x4, #-16]
  10:*  a9001c06        stp     x6, x7, [x0]            <-- trapping instruction

Code starting with the faulting instruction
===========================================
   0:   a9001c06        stp     x6, x7, [x0]
[26848.578934] SMP: stopping secondary CPUs
[26848.582664] Starting crashdump kernel...
[26848.583602] Bye!
Comment 2 Zorro Lang 2022-06-20 06:10:40 UTC
Same panic on another machine (s390x):

[10054.497558] run fstests generic/465 at 2022-06-19 16:09:21                    
[10055.731299] ================================================================= 
=                                                                                
[10055.731308] BUG: KASAN: use-after-free in _copy_to_iter+0x830/0x1030          
[10055.731324] Write of size 16 at addr 0000000090ebd000 by task nfsd/45999      
[10055.731328]                                                                   
[10055.731331] CPU: 1 PID: 45999 Comm: nfsd Kdump: loaded Not tainted 5.19.0-rc2 
+ #1                                                                             
[10055.731335] Hardware name: IBM 8561 LT1 400 (z/VM 7.2.0)                      
[10055.731338] Call Trace:                                                       
[10055.731339]  [<000000007bc24fda>] dump_stack_lvl+0xfa/0x150                   
[10055.731345]  [<000000007bc173bc>] print_address_description.constprop.0+0x64/ 
0x3a8                                                                            
[10055.731351]  [<000000007a98757e>] print_report+0xbe/0x230                     
[10055.731356]  [<000000007a987ba6>] kasan_report+0xa6/0x1e0                     
[10055.731359]  [<000000007a988fa4>] kasan_check_range+0x174/0x1c0               
[10055.731362]  [<000000007a989a38>] memcpy+0x58/0x90                            
[10055.731365]  [<000000007affd0c0>] _copy_to_iter+0x830/0x1030                  
[10055.731369]  [<000000007affddd0>] copy_page_to_iter+0x510/0xcb0               
[10055.731372]  [<000000007a7e986c>] filemap_read+0x52c/0x950                    
[10055.731378]  [<001bffff80599042>] xfs_file_buffered_read+0x1c2/0x410 [xfs]    
[10055.731751]  [<001bffff80599eba>] xfs_file_read_iter+0x28a/0x4c0 [xfs]        
[10055.731975]  [<000000007aa1084a>] do_iter_readv_writev+0x2ca/0x4c0            
[10055.731981]  [<000000007aa1102a>] do_iter_read+0x23a/0x3a0                    
[10055.731984]  [<001bffff80f58d30>] nfsd_readv+0x1e0/0x710 [nfsd]               
[10055.732070]  [<001bffff80fa2f88>] nfsd4_encode_read_plus_data+0x3a8/0x770 [nf 
sd]                                                                              
[10055.732129]  [<001bffff80fa5010>] nfsd4_encode_read_plus+0x3e0/0xaa0 [nfsd]   
[10055.732188]  [<001bffff80fbc0ac>] nfsd4_encode_operation+0x21c/0xab0 [nfsd]   
[10055.732249]  [<001bffff80f9ca7e>] nfsd4_proc_compound+0x125e/0x21a0 [nfsd]    
[10055.732307]  [<001bffff80f441aa>] nfsd_dispatch+0x44a/0xc40 [nfsd]            
[10055.732362]  [<001bffff80b8d00c>] svc_process_common+0x92c/0x1cd0 [sunrpc]    
[10055.732500]  [<001bffff80b8e6ac>] svc_process+0x2fc/0x4c0 [sunrpc]            
[10055.732579]  [<001bffff80f42f4e>] nfsd+0x31e/0x600 [nfsd]                     
[10055.732634]  [<000000007a2cc514>] kthread+0x2a4/0x360                         
[10055.732640]  [<000000007a186a5a>] __ret_from_fork+0x8a/0xf0                   
[10055.732645]  [<000000007bc5575a>] ret_from_fork+0xa/0x40                      
[10055.732650] 1 lock held by nfsd/45999:                                        
[10055.732653]  #0: 000000009cc7fb38 (&sb->s_type->i_mutex_key#13){++++}-{3:3},  
at: xfs_ilock+0x2fa/0x4e0 [xfs]                                                  
[10055.732887]                                                                   
[10055.732888] Allocated by task 601543:                                         
[10055.732890]  kasan_save_stack+0x34/0x60                                       
[10055.732893]  __kasan_slab_alloc+0x84/0xb0                                     
[10055.732896]  kmem_cache_alloc+0x1e2/0x3d0                                     
[10055.732900]  security_file_alloc+0x3a/0x150                                   
[10055.732906]  __alloc_file+0xc0/0x210                                          
[10055.732908]  alloc_empty_file+0x5c/0x140                                      
[10055.732911]  path_openat+0xf8/0x700                                           
[10055.732914]  do_filp_open+0x1b0/0x390                                         
[10055.732917]  do_sys_openat2+0x134/0x3c0                                       
[10055.732920]  do_sys_open+0xdc/0x120                                           
[10055.732922]  do_syscall+0x22c/0x330                                           
[10055.732925]  __do_syscall+0xce/0xf0                                           
[10055.732928]  system_call+0x82/0xb0                                            
[10055.732931]                                                                   
[10055.732932] Freed by task 601543:                                             
[10055.732933]  kasan_save_stack+0x34/0x60                                       
[10055.732935]  kasan_set_track+0x36/0x50                                        
[10055.732937]  kasan_set_free_info+0x34/0x60                                    
[10055.732940]  __kasan_slab_free+0x106/0x150                                    
[10055.732942]  slab_free_freelist_hook+0x148/0x230                              
[10055.732946]  kmem_cache_free+0x132/0x370                                      
[10055.732948]  __fput+0x2b2/0x700                                               
[10055.732950]  task_work_run+0xf4/0x1b0                                         
[10055.732952]  exit_to_user_mode_prepare+0x286/0x290                            
[10055.732957]  __do_syscall+0xce/0xf0                                           
[10055.732959]  system_call+0x82/0xb0                                            
[10055.732962]                                                                   
[10055.732962] The buggy address belongs to the object at 0000000090ebd000       
[10055.732962]  which belongs to the cache lsm_file_cache of size 16             
[10055.732965] The buggy address is located 0 bytes inside of                    
[10055.732965]  16-byte region [0000000090ebd000, 0000000090ebd010)              
[10055.732968]                                                                   
[10055.732969] The buggy address belongs to the physical page:                   
[10055.732970] page:00000000b4bd66d5 refcount:1 mapcount:0 mapping:0000000000000 
000 index:0x0 pfn:0x90ebd                                                        
[10055.732975] flags: 0x2000000000000200(slab|node=0|zone=1)                     
[10055.732982] raw: 2000000000000200 0000000000000100 0000000000000122 000000008 
024a200                                                                          
[10055.732985] raw: 0000000000000000 0080010000000000 ffffffff00000001 000000000 
0000000                                                                          
[10055.732986] page dumped because: kasan: bad access detected                   
[10055.732988]                                                                   
[10055.732989] Memory state around the buggy address:                            
[10055.732990]  0000000090ebcf00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0 
0                                                                                
[10055.732992]  0000000090ebcf80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
0                                                                                
[10055.732994] >0000000090ebd000: fa fb fc fc fa fb fc fc fa fb fc fc fa fb fc f 
c                                                                                
[10055.732995]                    ^                                              
[10055.732997]  0000000090ebd080: fa fb fc fc 00 00 fc fc fa fb fc fc 00 00 fc f 
c                                                                                
[10055.732999]  0000000090ebd100: 00 00 fc fc 00 00 fc fc fa fb fc fc fa fb fc f 
c                                                                                
[10055.733001] ================================================================= 
=                                                                                
[10055.733031] Disabling lock debugging due to kernel taint                      
[10058.081326] systemd-udevd (601251) used greatest stack depth: 45056 bytes lef 
t                                                                                
[10058.575324] Unable to handle kernel pointer dereference in virtual kernel add 
ress space                                                                       
[10058.575333] Failing address: 0185c58585858000 TEID: 0185c58585858803          
[10058.575337] Fault in home space mode while using kernel ASCE.                 
[10058.575342] AS:000000007d39400b R2:0000000000000028                           
[10058.575389] Oops: 0038 ilc:3 [#1] SMP                                         
[10058.575423] Modules linked in: tls rpcsec_gss_krb5 nfsv4 dns_resolver nfs fsc 
ache netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd gr 
ace loop lcs ctcm fsm zfcp scsi_transport_fc dasd_fba_mod rfkill vfio_ccw mdev v 
fio_iommu_type1 zcrypt_cex4 sunrpc vfio drm i2c_core fb fuse font drm_panel_orie 
ntation_quirks xfs libcrc32c ghash_s390 prng aes_s390 des_s390 sha3_512_s390 sha 
3_256_s390 qeth_l2 bridge stp llc dasd_eckd_mod dasd_mod qeth qdio ccwgroup dm_m 
irror dm_region_hash dm_log dm_mod pkey zcrypt                                   
[10058.575531] CPU: 1 PID: 754 Comm: systemd-journal Kdump: loaded Tainted: G    
 B             5.19.0-rc2+ #1                                                    
[10058.575540] Hardware name: IBM 8561 LT1 400 (z/VM 7.2.0)                      
[10058.575547] Krnl PSW : 0704e00180000000 000000007a989e3c (qlist_free_all+0x9c 
/0x130)                                                                          
[10058.575572]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI: 
0 EA:3                                                                           
[10058.575579] Krnl GPRS: 000000000098b130 0005002100000001 0185c58585858580 000 
000007c9111a8                                                                    
[10058.575584]            0000000091a8b000 0005002100000000 0000000091a8b000 001 
bff80018df5e8                                                                    
[10058.575588]            0000000000000000 0000000091a8b000 0000000080082e00 616 
1616161616161                                                                    
[10058.575592]            000000007c3cd090 000000007ab19aa6 000000007a989e1e 001 
bff80018df4e0                                                                    
[10058.575602] Krnl Code: 000000007a989e2a: c43800d22e97        lgrl    %r3,0000 
00007c3cfb58                                                                     
[10058.575602]            000000007a989e30: ec2b06b93a59        risbgn  %r2,%r11 
,6,185,58                                                                        
[10058.575602]           #000000007a989e36: e32030000008        ag      %r2,0(%r 
3)                                                                               
[10058.575602]           >000000007a989e3c: e33020080004        lg      %r3,8(%r 
2)                                                                               
[10058.575602]            000000007a989e42: a7310001            tmll    %r3,1    
[10058.575602]            000000007a989e46: a774003a            brc     7,000000 
007a989eba                                                                       
[10058.575602]            000000007a989e4a: e33020000004        lg      %r3,0(%r 
2)                                                                               
[10058.575602]            000000007a989e50: a7310200            tmll    %r3,512  
[10058.575635] Call Trace:                                                       
[10058.575638]  [<000000007a989e3c>] qlist_free_all+0x9c/0x130                   
[10058.575643] ([<000000007a989e1e>] qlist_free_all+0x7e/0x130)                  
[10058.575647]  [<000000007a98a45a>] kasan_quarantine_reduce+0x16a/0x1c0         
[10058.575652]  [<000000007a98720e>] __kasan_slab_alloc+0x9e/0xb0                
[10058.575657]  [<000000007a9810a4>] __kmalloc+0x214/0x440                       
[10058.575663]  [<000000007ab19aa6>] inotify_handle_inode_event+0x1b6/0x7d0      
[10058.575669]  [<000000007ab0ee74>] fsnotify_handle_inode_event.isra.0+0x1c4/0x 
2f0                                                                              
[10058.575674]  [<000000007ab0f490>] send_to_group+0x4f0/0x6c0                   
[10058.575678]  [<000000007ab0fe14>] fsnotify+0x654/0xb30                        
[10058.575682]  [<000000007ab10ca2>] __fsnotify_parent+0x372/0x780               
[10058.575687]  [<000000007aa7eb9e>] notify_change+0x96e/0xcf0                   
[10058.575693]  [<000000007aa0a0c8>] do_truncate+0x108/0x190                     
[10058.575699]  [<000000007aa0aafc>] do_sys_ftruncate+0x31c/0x600                
[10058.575703]  [<000000007a18da8c>] do_syscall+0x22c/0x330                      
[10058.575709]  [<000000007bc2cb6e>] __do_syscall+0xce/0xf0                      
[10058.575716]  [<000000007bc55722>] system_call+0x82/0xb0                       
[10058.575722] INFO: lockdep is turned off.                                      
[10058.575725] Last Breaking-Event-Address:                                      
[10058.575727]  [<000000007a985860>] ___cache_free+0x150/0x2a0                   
[10058.575733] ---[ end trace 0000000000000000 ]---                              
[10058.590086] systemd[1]: systemd-journald.service: Scheduled restart job, rest 
art counter is at 2.                                                             
[10058.590588] systemd[1]: Stopped Journal Service.                              
[10058.590758] systemd[1]: systemd-journald.service: Consumed 4.770s CPU time.   
[10058.596950] systemd[1]: Starting Journal Service...                           
[10058.634628] systemd-journald[601774]: File /run/log/journal/23dc967c665d48678 
d6de8983973d399/system.journal corrupted or uncleanly shut down, renaming and re 
placing.                                                                         
[-- MARK -- Sun Jun 19 20:10:00 2022] 
[10148.825091] systemd[1]: systemd-journald.service: start operation timed out.  
Terminating.                                                                     
[10180.285606] Unable to handle kernel pointer dereference in virtual kernel add 
ress space                                                                       
[10180.285615] Failing address: 0185c58585858000 TEID: 0185c58585858803          
[10180.285618] Fault in home space mode while using kernel ASCE.                 
[10180.285624] AS:000000007d39400b R2:0000000000000028                           
[10180.285671] Oops: 0038 ilc:3 [#2] SMP                                         
[10180.285707] Modules linked in: tls rpcsec_gss_krb5 nfsv4 dns_resolver nfs fsc 
ache netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd gr 
ace loop lcs ctcm fsm zfcp scsi_transport_fc dasd_fba_mod rfkill vfio_ccw mdev v 
fio_iommu_type1 zcrypt_cex4 sunrpc vfio drm i2c_core fb fuse font drm_panel_orie 
ntation_quirks xfs libcrc32c ghash_s390 prng aes_s390 des_s390 sha3_512_s390 sha 
3_256_s390 qeth_l2 bridge stp llc dasd_eckd_mod dasd_mod qeth qdio ccwgroup dm_m 
irror dm_region_hash dm_log dm_mod pkey zcrypt                                   
[10180.285815] CPU: 1 PID: 908 Comm: gmain Kdump: loaded Tainted: G    B D       
     5.19.0-rc2+ #1                                                              
[10180.285825] Hardware name: IBM 8561 LT1 400 (z/VM 7.2.0)                      
[10180.285833] Krnl PSW : 0704e00180000000 000000007a989e3c (qlist_free_all+0x9c 
/0x130)                                                                          
[10180.285858]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI: 
0 EA:3                                                                           
[10180.285864] Krnl GPRS: 0000000000000001 001c000000000000 0185c58585858580 000 
000007c9111a8                                                                    
[10180.285869]            0000000000000000 000000007a3bf8a2 000000009315c000 001 
bff8001f0fab8                                                                    
[10180.285873]            0000000000000000 000000009315c000 000000008026f200 616 
1616161616161                                                                    
[10180.285877]            000000007c3cd090 000000007c2f9f98 000000007a989e1e 001 
bff8001f0f9b0                                                                    
[10180.285888] Krnl Code: 000000007a989e2a: c43800d22e97        lgrl    %r3,0000 
00007c3cfb58                                                                     
[10180.285888]            000000007a989e30: ec2b06b93a59        risbgn  %r2,%r11 
,6,185,58                                                                        
[10180.285888]           #000000007a989e36: e32030000008        ag      %r2,0(%r 
3)                                                                               
[10180.285888]           >000000007a989e3c: e33020080004        lg      %r3,8(%r 
2)                                                                               
[10180.285888]            000000007a989e42: a7310001            tmll    %r3,1    
[10180.285888]            000000007a989e46: a774003a            brc     7,000000 
007a989eba                                                                       
[10180.285888]            000000007a989e4a: e33020000004        lg      %r3,0(%r 
2)                                                                               
[10180.285888]            000000007a989e50: a7310200            tmll    %r3,512  
[10180.285921] Call Trace:                                                       
[10180.285924]  [<000000007a989e3c>] qlist_free_all+0x9c/0x130                   
[10180.285929] ([<000000007a989e1e>] qlist_free_all+0x7e/0x130)                  
[10180.285933]  [<000000007a98a45a>] kasan_quarantine_reduce+0x16a/0x1c0         
[10180.285938]  [<000000007a98720e>] __kasan_slab_alloc+0x9e/0xb0                
[10180.285943]  [<000000007a982102>] kmem_cache_alloc+0x1e2/0x3d0                
[10180.285949]  [<000000007aa4e9d6>] getname_flags.part.0+0x56/0x430             
[10180.285955]  [<000000007aa5073a>] user_path_at_empty+0x3a/0x80                
[10180.285959]  [<000000007ab1b59a>] inotify_find_inode+0x3a/0x150               
[10180.285966]  [<000000007ab1c9de>] __s390x_sys_inotify_add_watch+0x17e/0x2c0   
[10180.285971]  [<000000007a18da8c>] do_syscall+0x22c/0x330                      
[10180.285978]  [<000000007bc2cb6e>] __do_syscall+0xce/0xf0                      
[10180.285984]  [<000000007bc55722>] system_call+0x82/0xb0                       
[10180.285990] INFO: lockdep is turned off.                                      
[10180.285993] Last Breaking-Event-Address:                                      
[10180.285995]  [<000000007a985860>] ___cache_free+0x150/0x2a0                   
[10180.286001] ---[ end trace 0000000000000000 ]---
Comment 3 Dave Chinner 2022-06-23 23:34:42 UTC
On Mon, Jun 20, 2022 at 06:10:40AM +0000, bugzilla-daemon@kernel.org wrote:
> https://bugzilla.kernel.org/show_bug.cgi?id=216151
> 
> --- Comment #2 from Zorro Lang (zlang@redhat.com) ---
> Same panic on another machine (s390x):
> 
> [10054.497558] run fstests generic/465 at 2022-06-19 16:09:21                 
> [10055.731299]
> ================================================================= 
> =                                                                             
> [10055.731308] BUG: KASAN: use-after-free in _copy_to_iter+0x830/0x1030       
> [10055.731324] Write of size 16 at addr 0000000090ebd000 by task nfsd/45999   
> [10055.731328]                                                                
> [10055.731331] CPU: 1 PID: 45999 Comm: nfsd Kdump: loaded Not tainted
> 5.19.0-rc2 
> + #1                                                                          
> [10055.731335] Hardware name: IBM 8561 LT1 400 (z/VM 7.2.0)                   
> [10055.731338] Call Trace:                                                    
> [10055.731339]  [<000000007bc24fda>] dump_stack_lvl+0xfa/0x150                
> [10055.731345]  [<000000007bc173bc>]
> print_address_description.constprop.0+0x64/ 
> 0x3a8                                                                         
> [10055.731351]  [<000000007a98757e>] print_report+0xbe/0x230                  
> [10055.731356]  [<000000007a987ba6>] kasan_report+0xa6/0x1e0                  
> [10055.731359]  [<000000007a988fa4>] kasan_check_range+0x174/0x1c0            
> [10055.731362]  [<000000007a989a38>] memcpy+0x58/0x90                         
> [10055.731365]  [<000000007affd0c0>] _copy_to_iter+0x830/0x1030               
> [10055.731369]  [<000000007affddd0>] copy_page_to_iter+0x510/0xcb0            
> [10055.731372]  [<000000007a7e986c>] filemap_read+0x52c/0x950                 
> [10055.731378]  [<001bffff80599042>] xfs_file_buffered_read+0x1c2/0x410 [xfs] 
> [10055.731751]  [<001bffff80599eba>] xfs_file_read_iter+0x28a/0x4c0 [xfs]     
> [10055.731975]  [<000000007aa1084a>] do_iter_readv_writev+0x2ca/0x4c0         
> [10055.731981]  [<000000007aa1102a>] do_iter_read+0x23a/0x3a0                 
> [10055.731984]  [<001bffff80f58d30>] nfsd_readv+0x1e0/0x710 [nfsd]            
> [10055.732070]  [<001bffff80fa2f88>] nfsd4_encode_read_plus_data+0x3a8/0x770
> [nf 
> sd]                                                                           
> [10055.732129]  [<001bffff80fa5010>] nfsd4_encode_read_plus+0x3e0/0xaa0
> [nfsd]  
> [10055.732188]  [<001bffff80fbc0ac>] nfsd4_encode_operation+0x21c/0xab0
> [nfsd]  
> [10055.732249]  [<001bffff80f9ca7e>] nfsd4_proc_compound+0x125e/0x21a0 [nfsd] 
> [10055.732307]  [<001bffff80f441aa>] nfsd_dispatch+0x44a/0xc40 [nfsd]         
> [10055.732362]  [<001bffff80b8d00c>] svc_process_common+0x92c/0x1cd0 [sunrpc] 
> [10055.732500]  [<001bffff80b8e6ac>] svc_process+0x2fc/0x4c0 [sunrpc]         
> [10055.732579]  [<001bffff80f42f4e>] nfsd+0x31e/0x600 [nfsd]                  
> [10055.732634]  [<000000007a2cc514>] kthread+0x2a4/0x360                      
> [10055.732640]  [<000000007a186a5a>] __ret_from_fork+0x8a/0xf0                
> [10055.732645]  [<000000007bc5575a>] ret_from_fork+0xa/0x40                   

This doesn't look like an XFS problem. The _copy_to_iter() call that
is tripping up here is copying from the page cache page to the
buffer supplied to XFS by the NFSD in the iov_iter structure. We
know that because it's a memory write operation that is triggering
(read from page cache page, write to iov_iter buffer) here.

> [10055.732650] 1 lock held by nfsd/45999:                                     
> [10055.732653]  #0: 000000009cc7fb38
> (&sb->s_type->i_mutex_key#13){++++}-{3:3}, 
> at: xfs_ilock+0x2fa/0x4e0 [xfs]                                               
> [10055.732887]                                                                
> [10055.732888] Allocated by task 601543:                                      
> [10055.732890]  kasan_save_stack+0x34/0x60                                    
> [10055.732893]  __kasan_slab_alloc+0x84/0xb0                                  
> [10055.732896]  kmem_cache_alloc+0x1e2/0x3d0                                  
> [10055.732900]  security_file_alloc+0x3a/0x150                                
> [10055.732906]  __alloc_file+0xc0/0x210                                       
> [10055.732908]  alloc_empty_file+0x5c/0x140                                   
> [10055.732911]  path_openat+0xf8/0x700                                        
> [10055.732914]  do_filp_open+0x1b0/0x390                                      
> [10055.732917]  do_sys_openat2+0x134/0x3c0                                    
> [10055.732920]  do_sys_open+0xdc/0x120                                        
> [10055.732922]  do_syscall+0x22c/0x330                                        
> [10055.732925]  __do_syscall+0xce/0xf0                                        
> [10055.732928]  system_call+0x82/0xb0                                         
> [10055.732931]                                                                
> [10055.732932] Freed by task 601543:                                          
> [10055.732933]  kasan_save_stack+0x34/0x60                                    
> [10055.732935]  kasan_set_track+0x36/0x50                                     
> [10055.732937]  kasan_set_free_info+0x34/0x60                                 
> [10055.732940]  __kasan_slab_free+0x106/0x150                                 
> [10055.732942]  slab_free_freelist_hook+0x148/0x230                           
> [10055.732946]  kmem_cache_free+0x132/0x370                                   
> [10055.732948]  __fput+0x2b2/0x700                                            
> [10055.732950]  task_work_run+0xf4/0x1b0                                      
> [10055.732952]  exit_to_user_mode_prepare+0x286/0x290                         
> [10055.732957]  __do_syscall+0xce/0xf0                                        
> [10055.732959]  system_call+0x82/0xb0                                         

And that memory was last used as a struct file *, again something
that XFS does not allocate but will be allocated by the NFSD as it
opens and closes the files it receives requests to process for...

> [10058.575635] Call Trace:                                                    
> [10058.575638]  [<000000007a989e3c>] qlist_free_all+0x9c/0x130                
> [10058.575643] ([<000000007a989e1e>] qlist_free_all+0x7e/0x130)               
> [10058.575647]  [<000000007a98a45a>] kasan_quarantine_reduce+0x16a/0x1c0      
> [10058.575652]  [<000000007a98720e>] __kasan_slab_alloc+0x9e/0xb0             
> [10058.575657]  [<000000007a9810a4>] __kmalloc+0x214/0x440                    
> [10058.575663]  [<000000007ab19aa6>] inotify_handle_inode_event+0x1b6/0x7d0   
> [10058.575669]  [<000000007ab0ee74>]
> fsnotify_handle_inode_event.isra.0+0x1c4/0x 
> 2f0                                                                           
> [10058.575674]  [<000000007ab0f490>] send_to_group+0x4f0/0x6c0                
> [10058.575678]  [<000000007ab0fe14>] fsnotify+0x654/0xb30                     
> [10058.575682]  [<000000007ab10ca2>] __fsnotify_parent+0x372/0x780            
> [10058.575687]  [<000000007aa7eb9e>] notify_change+0x96e/0xcf0                
> [10058.575693]  [<000000007aa0a0c8>] do_truncate+0x108/0x190                  
> [10058.575699]  [<000000007aa0aafc>] do_sys_ftruncate+0x31c/0x600             
> [10058.575703]  [<000000007a18da8c>] do_syscall+0x22c/0x330                   
> [10058.575709]  [<000000007bc2cb6e>] __do_syscall+0xce/0xf0                   
> [10058.575716]  [<000000007bc55722>] system_call+0x82/0xb0                    
> [10058.575722] INFO: lockdep is turned off.                                   
> [10058.575725] Last Breaking-Event-Address:                                   
> [10058.575727]  [<000000007a985860>] ___cache_free+0x150/0x2a0                
> [10058.575733] ---[ end trace 0000000000000000 ]---                           

And this subsequent oops has doesn't have anything to do with XFS
either - this is indicative of slab cache (memory heap) corruption
causing stuff to go badly wrong.

Hence I think XFS is messenger here - something is corrupting the
heap and an NFSD->XFS code path is the first to trip over it.

Cheers,

Dave.
Comment 4 Chuck Lever 2022-06-26 21:04:10 UTC
You can disable the client's use of NFSv4.2's READ_PLUS operation:

209 config NFS_V4_2_READ_PLUS
210         bool "NFS: Enable support for the NFSv4.2 READ_PLUS operation"
211         depends on NFS_V4_2
212         default n
213         help
214          This is intended for developers only. The READ_PLUS operation has
215          been shown to have issues under specific conditions and should not
216          be used in production.

As an experiment to see if the problem goes away.
Comment 5 Chuck Lever 2022-07-04 16:21:54 UTC
Commit a23dd544debc ("SUNRPC: Fix READ_PLUS crasher"), which addresses this issue, appears in v5.19-rc5.