On Fedora rawhide system with kernel 5.19.0-0.rc0.20220526gitbabf0bb978e3.4.fc37.x86_64, which base on latest upstream mailine linux which HEAD is: babf0bb978e3 Merge tag 'xfs-5.19-for-linus' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux I hit a bug with DAX testing on xfs. Not sure if it's a bug from XFS side or mm side. Report to xfs list at first, feel free to change it to other component if it's not a xfs/iomap bug. # ./check generic/623 generic/139 generic/591 generic/506 FSTYP -- xfs (non-debug) PLATFORM -- Linux/x86_64 hp-xxxxxx-xx 5.19.0-0.rc0.20220526gitbabf0bb978e3.4.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 26 16:02:31 UTC 2022 MKFS_OPTIONS -- -f -m reflink=0 /dev/pmem0p2 MOUNT_OPTIONS -- -o dax=always -o context=system_u:object_r:root_t:s0 /dev/pmem0p2 /mnt/scratch generic/139 5s ... [not run] Reflink not supported by test filesystem type: xfs generic/506 6s ... 5s generic/591 3s ... 2s generic/623 4s ... ^C^C # dmesg [91876.709062] run fstests generic/623 at 2022-05-29 17:32:56 [91877.213522] systemd[1]: Started fstests-generic-623.scope - /usr/bin/bash -c test -w /proc/self/oom_score_adj && echo 250 > /proc/self/oom_score_adj; exec ./tests/generic/623. [91878.417157] XFS (pmem0p2): DAX enabled. Warning: EXPERIMENTAL, use at your own risk [91878.417443] XFS (pmem0p2): Mounting V5 Filesystem [91878.486455] XFS (pmem0p2): Ending clean mount [91878.497348] XFS (pmem0p2): User initiated shutdown received. [91878.497467] XFS (pmem0p2): Metadata I/O Error (0x4) detected at xfs_fs_goingdown+0x6b/0xa0 [xfs] (fs/xfs/xfs_fsops.c:485). Shutting down filesystem. [91878.497651] XFS (pmem0p2): Please unmount the filesystem and rectify the problem(s) [91878.513406] systemd[1]: mnt-scratch.mount: Deactivated successfully. [91878.534953] XFS (pmem0p2): Unmounting Filesystem [91878.966315] XFS (pmem0p2): DAX enabled. Warning: EXPERIMENTAL, use at your own risk [91878.966650] XFS (pmem0p2): Mounting V5 Filesystem [91878.979236] XFS (pmem0p2): Ending clean mount [91879.014998] ------------[ cut here ]------------ [91879.015001] kernel BUG at mm/page_table_check.c:51! [91879.015012] invalid opcode: 0000 [#1] PREEMPT SMP PTI [91879.015019] CPU: 12 PID: 48184 Comm: xfs_io Tainted: G S I -------- --- 5.19.0-0.rc0.20220526gitbabf0bb978e3.4.fc37.x86_64 #1 [91879.015022] Hardware name: HP ProLiant DL380p Gen8, BIOS P70 08/02/2014 [91879.015024] RIP: 0010:page_table_check_set.part.0+0x89/0xe0 [91879.015037] Code: 75 64 44 89 c1 f0 0f c1 08 83 c1 01 83 f9 01 7e 04 84 db 75 67 48 83 c6 01 48 03 15 41 a4 e6 01 4c 39 e6 74 4f 48 85 d2 75 c2 <0f> 0b f7 c5 ff 0f 00 00 75 a4 48 8b 45 00 a9 00 00 01 00 74 99 48 [91879.015040] RSP: 0000:ffffbdb2a1437b30 EFLAGS: 00010246 [91879.015044] RAX: ffff970cc3dbdd30 RBX: 0000000000000001 RCX: 0000000000000000 [91879.015047] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000 [91879.015048] RBP: fffff49809008300 R08: 0000000000000000 R09: 0000000000000000 [91879.015050] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000001 [91879.015052] R13: fffff49809008300 R14: ffff9711fe8d7bb0 R15: ffff971191976200 [91879.015054] FS: 00007fceec2f4740(0000) GS:ffff9714eae00000(0000) knlGS:0000000000000000 [91879.015056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [91879.015058] CR2: 00007fceec576000 CR3: 000000053742a001 CR4: 00000000001706e0 [91879.015061] Call Trace: [91879.015063] <TASK> [91879.015066] insert_pfn+0x10e/0x160 [91879.015074] __vm_insert_mixed+0xb0/0xd0 [91879.015079] dax_fault_iter+0x742/0xa40 [91879.015088] ? lock_is_held_type+0xd0/0x140 [91879.015101] dax_iomap_pte_fault+0x1c9/0x640 [91879.015113] __xfs_filemap_fault+0x305/0x410 [xfs] [91879.015265] __do_fault+0x36/0x1a0 [91879.015270] __handle_mm_fault+0xc66/0x1470 [91879.015277] handle_mm_fault+0x11a/0x3a0 [91879.015282] do_user_addr_fault+0x1e0/0x6a0 [91879.015292] exc_page_fault+0x77/0x2d0 [91879.015297] asm_exc_page_fault+0x27/0x30 [91879.015304] RIP: 0033:0x558d1bd1488e [91879.015330] Code: c0 0f 84 e1 00 00 00 48 8b 05 8e c2 02 00 48 2b 58 10 49 8d 14 1c 45 85 f6 75 55 4d 85 e4 0f 8e c7 fe ff ff 48 8b 00 44 89 ee <44> 88 2c 18 48 8d 43 01 49 83 fc 01 0f 8e af fe ff ff 48 8b 0d 59 [91879.015333] RSP: 002b:00007ffd5dc58990 EFLAGS: 00010206 [91879.015336] RAX: 00007fceec576000 RBX: 0000000000000000 RCX: 0000000000001000 [91879.015338] RDX: 0000000000001000 RSI: 0000000000000058 RDI: 0000000000000000 [91879.015340] RBP: 0000558d1c2e83e0 R08: 1999999999999999 R09: 0000000000000000 [91879.015342] R10: 00007fceec4a0ac0 R11: 00007fceec4a13c0 R12: 0000000000001000 [91879.015343] R13: 0000000000000058 R14: 0000000000000000 R15: 0000000000001000 [91879.015350] </TASK> [91879.015352] Modules linked in: scsi_debug nls_utf8 hfsplus hfs vfat fat isofs binfmt_misc tls dm_dust nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_rejec t_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack iTCO_wdt nf_defrag_ipv6 intel_pmc_bxt nf_defrag_ipv4 intel_rapl_msr iTCO_vendor_support intel_rapl_common sb_edac x86_pkg_temp_thermal ip_set intel_powerclamp coretemp rfkill nf_tables nfnetlink qrtr kvm_intel kvm irqbypass rapl sunrpc intel_cstate lpc_ich hpilo ipmi_ssif pktcdvd intel_uncore dax_pmem acpi_ipmi ioatdma tg 3 dca ipmi_si acpi_power_meter fuse zram xfs nd_pmem nd_btt crct10dif_pclmul crc32_pclmul crc32c_intel hpsa nd_e820 libnvdimm ghash_clmulni_intel serio_raw mgag200 scsi_transport_sas hpwdt at a_generic pata_acpi scsi_dh_rdac scsi_dh_emc scsi_dh_alua dm_multipath ipmi_devintf ipmi_msghandler [91879.015432] ---[ end trace 0000000000000000 ]--- [91879.042514] RIP: 0010:page_table_check_set.part.0+0x89/0xe0 [91879.042522] Code: 75 64 44 89 c1 f0 0f c1 08 83 c1 01 83 f9 01 7e 04 84 db 75 67 48 83 c6 01 48 03 15 41 a4 e6 01 4c 39 e6 74 4f 48 85 d2 75 c2 <0f> 0b f7 c5 ff 0f 00 00 75 a4 48 8b 45 00 a9 00 00 01 00 74 99 48 [91879.042524] RSP: 0000:ffffbdb2a1437b30 EFLAGS: 00010246 [91879.042527] RAX: ffff970cc3dbdd30 RBX: 0000000000000001 RCX: 0000000000000000 [91879.042528] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000 [91879.042529] RBP: fffff49809008300 R08: 0000000000000000 R09: 0000000000000000 [91879.042531] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000001 [91879.042532] R13: fffff49809008300 R14: ffff9711fe8d7bb0 R15: ffff971191976200 [91879.042534] FS: 00007fceec2f4740(0000) GS:ffff9714eae00000(0000) knlGS:0000000000000000 [91879.042536] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [91879.042537] CR2: 00007fceec576000 CR3: 000000053742a001 CR4: 00000000001706e0 [91879.042539] note: xfs_io[48184] exited with preempt_count 1 [91882.423346] systemd[1]: Created slice system-dbus\x2d:1.2\x2dorg.freedesktop.problems.slice - Slice /system/dbus-:1.2-org.freedesktop.problems. [91882.429366] systemd[1]: Started dbus-:1.2-org.freedesktop.problems@0.service [92015.775618] systemd[1]: dbus-:1.2-org.freedesktop.problems@0.service: Deactivated successfully. [92332.353087] systemd[1]: Starting dnf-makecache.service - dnf makecache... [92347.887809] systemd[1]: dnf-makecache.service: Deactivated successfully. [92347.934938] systemd[1]: Finished dnf-makecache.service - dnf makecache. [92347.935805] systemd[1]: dnf-makecache.service: Consumed 6.327s CPU time. [92397.345084] sysrq: Show Blocked State [92397.365912] task:xfs_io state:D stack:14528 pid:48187 ppid: 47965 flags:0x00004006 [92397.408219] Call Trace: [92397.420926] <TASK> [92397.430899] __schedule+0x492/0x1640 [92397.447320] ? lock_acquire+0x26a/0x2d0 [92397.464582] ? rcu_read_lock_sched_held+0x10/0x70 [92397.485899] ? lock_release+0x215/0x460 [92397.503192] schedule+0x4e/0xb0 [92397.517338] rwsem_down_write_slowpath+0x35a/0x710 [92397.538897] down_write+0xad/0x110 [92397.554207] exit_mmap+0x46/0x1a0 [92397.569167] ? uprobe_clear_state+0x25/0x120 [92397.588450] ? __mutex_unlock_slowpath+0x2a/0x260 [92397.610111] ? uprobe_clear_state+0x68/0x120 [92397.630130] mmput+0x71/0x150 [92397.643507] do_exit+0x324/0xc40 [92397.658063] ? rcu_read_lock_sched_held+0x10/0x70 [92397.679239] do_group_exit+0x33/0xb0 [92397.695338] get_signal+0xbbc/0xbc0 [92397.711029] arch_do_signal_or_restart+0x30/0x770 [92397.732360] ? __schedule+0x49a/0x1640 [92397.749793] ? lock_is_held_type+0xe8/0x140 [92397.768690] exit_to_user_mode_prepare+0x172/0x270 [92397.790278] syscall_exit_to_user_mode+0x16/0x50 [92397.811059] do_syscall_64+0x67/0x80 [92397.827127] ? sched_clock_cpu+0xb/0xb0 [92397.844438] ? lock_release+0x14f/0x460 [92397.861976] ? _raw_spin_unlock_irq+0x24/0x50 [92397.883664] ? lock_is_held_type+0xe8/0x140 [92397.904426] ? do_syscall_64+0x67/0x80 [92397.922912] ? lockdep_hardirqs_on+0x7d/0x100 [92397.944397] ? do_syscall_64+0x67/0x80 [92397.961255] ? lockdep_hardirqs_on+0x7d/0x100 [92397.981189] entry_SYSCALL_64_after_hwframe+0x46/0xb0 [92398.004672] RIP: 0033:0x7fceec3fc422 [92398.020394] RSP: 002b:00007fceec2f2e30 EFLAGS: 00000293 ORIG_RAX: 0000000000000022 [92398.054524] RAX: fffffffffffffdfe RBX: 00007fceec2f3640 RCX: 00007fceec3fc422 [92398.086955] RDX: 0000000000000002 RSI: 0000000000000000 RDI: 0000000000000000 [92398.119583] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007ffd5dc5891f [92398.151800] R10: 0000000000000008 R11: 0000000000000293 R12: ffffffffffffff80 [92398.183959] R13: 0000000000000016 R14: 00007ffd5dc58820 R15: 00007fceebaf3000 [92398.216164] </TASK>
I just tried the lastest mainline upstream linux, it's 100% reproducible for me. And it's not only reproducible on xfs, ext4 with DAX can reproduce it too[1]. So it might be a common bug from mm? [1] [ 291.434410] run fstests generic/623 at 2022-05-30 15:14:32 [ 291.768382] systemd[1]: Started fstests-generic-623.scope - /usr/bin/bash -c test -w /proc/self/oom_score_adj && echo 250 > /proc/self/oom_score_adj; exec ./tests/generic/623. [ 294.406254] EXT4-fs (pmem0p2): mounted filesystem with ordered data mode. Quota mode: none. [ 294.442928] EXT4-fs (pmem0p2): shut down requested (1) [ 294.443089] Aborting journal on device pmem0p2-8. [ 294.477893] EXT4-fs (pmem0p2): unmounting filesystem. [ 294.479943] systemd[1]: mnt-scratch.mount: Deactivated successfully. [ 294.960374] EXT4-fs (pmem0p2): mounted filesystem with ordered data mode. Quota mode: none. [ 295.043716] ------------[ cut here ]------------ [ 295.043722] kernel BUG at mm/page_table_check.c:51! [ 295.043736] invalid opcode: 0000 [#1] PREEMPT SMP KASAN PTI [ 295.112220] CPU: 23 PID: 1936 Comm: xfs_io Tainted: G S I 5.18.0-mainline+ #2 [ 295.149570] Hardware name: HP ProLiant DL380p Gen8, BIOS P70 08/02/2014 [ 295.180088] RIP: 0010:page_table_check_set.part.0+0x184/0x300 [ 295.208336] Code: 48 8b 04 24 80 38 00 0f 85 2b 01 00 00 48 83 c3 01 4c 03 3d 5e e9 5a 03 4c 39 eb 0f 84 05 01 00 00 4d 85 ff 0f 85 53 ff ff ff <0f> 0b f7 c3 ff 0f 00 00 0f 85 f5 fe ff ff be 08 00 00 00 48 89 df [ 295.293142] RSP: 0000:ffffc9000d50f708 EFLAGS: 00010246 [ 295.316834] RAX: dffffc0000000000 RBX: ffffea00092180c0 RCX: ffffffff85a98a25 [ 295.349059] RDX: 1ffffd400124301b RSI: 0000000000000001 RDI: ffffea00092180d8 [ 295.381191] RBP: ffffea00092180c0 R08: 0000000000000000 R09: ffff88801bfa7607 [ 295.413394] R10: ffffed10037f4ec0 R11: 0000000000000005 R12: 0000000000000000 [ 295.445718] R13: 0000000000000001 R14: ffff88801bfa7600 R15: 0000000000000000 [ 295.477844] FS: 00007f590a83f740(0000) GS:ffff888085000000(0000) knlGS:0000000000000000 [ 295.518073] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 295.547270] CR2: 00007f590aac1000 CR3: 0000000049404003 CR4: 00000000001706e0 [ 295.581391] Call Trace: [ 295.602075] __page_table_check_pte_set+0x28f/0x350 [ 295.624144] ? __page_table_check_pte_clear+0x2b0/0x2b0 [ 295.647669] ? percpu_ref_put_many.constprop.0+0x1a0/0x1a0 [ 295.672391] ? __get_locked_pte+0x1b8/0x2e0 [ 295.691432] insert_pfn+0x22d/0x340 [ 295.707093] ? vm_map_pages_zero+0x10/0x10 [ 295.725738] ? pfn_modify_allowed+0x64/0x2b0 [ 295.745392] ? track_pfn_remap+0x1d0/0x1d0 [ 295.764201] __vm_insert_mixed+0x17f/0x1e0 [ 295.782965] ? vmf_insert_pfn+0x60/0x60 [ 295.800382] ? __dax_invalidate_entry+0x1c0/0x1c0 [ 295.822137] ? dax_direct_access+0x111/0x1b0 [ 295.841356] dax_fault_iter+0x100f/0x1bf0 [ 295.859436] ? grab_mapping_entry+0x4e0/0x4e0 [ 295.879076] ? iomap_iter+0xa02/0x10a0 [ 295.895964] dax_iomap_pte_fault+0x3f4/0xb80 [ 295.915252] ? dax_writeback_mapping_range+0xeb0/0xeb0 [ 295.938698] ? __ext4_journal_start_sb+0x345/0x460 [ 295.960264] ext4_dax_huge_fault+0x44f/0x940 [ 295.979436] ? ext4_file_open+0xa50/0xa50 [ 295.998477] ? var_wake_function+0x260/0x260 [ 296.020138] __do_fault+0xf8/0x4e0 [ 296.037077] do_fault+0x778/0xee0 [ 296.053492] ? restore_exclusive_pte+0x5d0/0x5d0 [ 296.076166] __handle_mm_fault+0xf82/0x26d0 [ 296.095032] ? vm_iomap_memory+0x140/0x140 [ 296.113504] ? count_memcg_events.constprop.0+0x40/0x50 [ 296.137317] handle_mm_fault+0x20e/0x750 [ 296.154958] do_user_addr_fault+0x345/0xd70 [ 296.173803] ? rcu_read_lock_sched_held+0x3c/0x70 [ 296.194972] exc_page_fault+0x65/0x100 [ 296.213635] asm_exc_page_fault+0x27/0x30 [ 296.232854] RIP: 0033:0x55d4406da88e [ 296.249860] Code: c0 0f 84 e1 00 00 00 48 8b 05 8e c2 02 00 48 2b 58 10 49 8d 14 1c 45 85 f6 75 55 4d 85 e4 0f 8e c7 fe ff ff 48 8b 00 44 89 ee <44> 88 2c 18 48 8d 43 01 49 83 fc 01 0f 8e af fe ff ff 48 8b 0d 59 [ 296.334980] RSP: 002b:00007fff29d58370 EFLAGS: 00010206 [ 296.358575] RAX: 00007f590aac1000 RBX: 0000000000000000 RCX: 0000000000001000 [ 296.390927] RDX: 0000000000001000 RSI: 0000000000000058 RDI: 0000000000000000 [ 296.423124] RBP: 000055d441b093e0 R08: 1999999999999999 R09: 0000000000000000 [ 296.455428] R10: 00007f590a9ebac0 R11: 00007f590a9ec3c0 R12: 0000000000001000 [ 296.489563] R13: 0000000000000058 R14: 0000000000000000 R15: 0000000000000200 [ 296.524615] </TASK> [ 296.535495] Modules linked in: nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ip v6 iTCO_wdt intel_rapl_msr intel_pmc_bxt nf_defrag_ipv4 iTCO_vendor_support intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp ip_set coretemp kvm_intel kvm irqbypass nf_tables rfkill rapl intel_cstate nfnetlink qrtr intel_uncore dax_pmem pcspkr sunrpc lpc_ich hpilo pktcdvd ipmi_ssif acpi_ipmi tg3 ioatdma ipmi_si acpi_power_meter dca fuse zram xfs nd_pmem nd_btt crct 10dif_pclmul crc32_pclmul nd_e820 crc32c_intel libnvdimm ghash_clmulni_intel hpsa serio_raw mgag200 hpwdt scsi_transport_sas ata_generic pata_acpi scsi_dh_rdac scsi_dh_emc scsi_dh_alua dm_mul tipath ipmi_devintf ipmi_msghandler [ 296.859756] ---[ end trace 0000000000000000 ]--- [ 296.897217] RIP: 0010:page_table_check_set.part.0+0x184/0x300 [ 296.897228] Code: 48 8b 04 24 80 38 00 0f 85 2b 01 00 00 48 83 c3 01 4c 03 3d 5e e9 5a 03 4c 39 eb 0f 84 05 01 00 00 4d 85 ff 0f 85 53 ff ff ff <0f> 0b f7 c3 ff 0f 00 00 0f 85 f5 fe ff ff be 08 00 00 00 48 89 df [ 296.897233] RSP: 0000:ffffc9000d50f708 EFLAGS: 00010246 [ 296.897238] RAX: dffffc0000000000 RBX: ffffea00092180c0 RCX: ffffffff85a98a25 [ 296.897241] RDX: 1ffffd400124301b RSI: 0000000000000001 RDI: ffffea00092180d8 [ 296.897244] RBP: ffffea00092180c0 R08: 0000000000000000 R09: ffff88801bfa7607 [ 296.897247] R10: ffffed10037f4ec0 R11: 0000000000000005 R12: 0000000000000000 [ 296.897249] R13: 0000000000000001 R14: ffff88801bfa7600 R15: 0000000000000000 [ 296.897253] FS: 00007f590a83f740(0000) GS:ffff888085000000(0000) knlGS:0000000000000000 [ 296.897256] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 296.897260] CR2: 00007f590aac1000 CR3: 0000000049404003 CR4: 00000000001706e0 [ 296.897264] note: xfs_io[1936] exited with preempt_count 1
Created attachment 301076 [details] kernel config file
Created attachment 301077 [details] unreproducible kernel config file