Bug 216047 - [generic/623 DAX] kernel BUG at mm/page_table_check.c:51!
Summary: [generic/623 DAX] kernel BUG at mm/page_table_check.c:51!
Status: NEW
Alias: None
Product: Memory Management
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: FileSystem/XFS Default Virtual Assignee
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-05-29 09:46 UTC by Zorro Lang
Modified: 2022-05-30 18:28 UTC (History)
0 users

See Also:
Kernel Version: 5.19.0-0.rc0
Subsystem:
Regression: No
Bisected commit-id:


Attachments
kernel config file (238.97 KB, text/plain)
2022-05-30 15:18 UTC, Zorro Lang
Details
unreproducible kernel config file (214.54 KB, text/plain)
2022-05-30 18:28 UTC, Zorro Lang
Details

Description Zorro Lang 2022-05-29 09:46:27 UTC
On Fedora rawhide system with kernel 5.19.0-0.rc0.20220526gitbabf0bb978e3.4.fc37.x86_64, which base on latest upstream mailine linux which HEAD is:
babf0bb978e3 Merge tag 'xfs-5.19-for-linus' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux

I hit a bug with DAX testing on xfs. Not sure if it's a bug from XFS side or mm side. Report to xfs list at first, feel free to change it to other component if it's not a xfs/iomap bug.

# ./check generic/623 generic/139 generic/591 generic/506
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 hp-xxxxxx-xx 5.19.0-0.rc0.20220526gitbabf0bb978e3.4.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 26 16:02:31 UTC 2022
MKFS_OPTIONS  -- -f -m reflink=0 /dev/pmem0p2
MOUNT_OPTIONS -- -o dax=always -o context=system_u:object_r:root_t:s0 /dev/pmem0p2 /mnt/scratch

generic/139 5s ... [not run] Reflink not supported by test filesystem type: xfs
generic/506 6s ...  5s
generic/591 3s ...  2s
generic/623 4s ... ^C^C

# dmesg
[91876.709062] run fstests generic/623 at 2022-05-29 17:32:56
[91877.213522] systemd[1]: Started fstests-generic-623.scope - /usr/bin/bash -c test -w /proc/self/oom_score_adj && echo 250 > /proc/self/oom_score_adj; exec ./tests/generic/623.
[91878.417157] XFS (pmem0p2): DAX enabled. Warning: EXPERIMENTAL, use at your own risk
[91878.417443] XFS (pmem0p2): Mounting V5 Filesystem
[91878.486455] XFS (pmem0p2): Ending clean mount
[91878.497348] XFS (pmem0p2): User initiated shutdown received.
[91878.497467] XFS (pmem0p2): Metadata I/O Error (0x4) detected at xfs_fs_goingdown+0x6b/0xa0 [xfs] (fs/xfs/xfs_fsops.c:485).  Shutting down filesystem.
[91878.497651] XFS (pmem0p2): Please unmount the filesystem and rectify the problem(s)
[91878.513406] systemd[1]: mnt-scratch.mount: Deactivated successfully.
[91878.534953] XFS (pmem0p2): Unmounting Filesystem
[91878.966315] XFS (pmem0p2): DAX enabled. Warning: EXPERIMENTAL, use at your own risk
[91878.966650] XFS (pmem0p2): Mounting V5 Filesystem
[91878.979236] XFS (pmem0p2): Ending clean mount
[91879.014998] ------------[ cut here ]------------
[91879.015001] kernel BUG at mm/page_table_check.c:51!
[91879.015012] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[91879.015019] CPU: 12 PID: 48184 Comm: xfs_io Tainted: G S        I      --------  ---  5.19.0-0.rc0.20220526gitbabf0bb978e3.4.fc37.x86_64 #1
[91879.015022] Hardware name: HP ProLiant DL380p Gen8, BIOS P70 08/02/2014
[91879.015024] RIP: 0010:page_table_check_set.part.0+0x89/0xe0
[91879.015037] Code: 75 64 44 89 c1 f0 0f c1 08 83 c1 01 83 f9 01 7e 04 84 db 75 67 48 83 c6 01 48 03 15 41 a4 e6 01 4c 39 e6 74 4f 48 85 d2 75 c2 <0f> 0b f7 c5 ff 0f 00 00 75 a4 48 8b 45 00
a9 00 00 01 00 74 99 48
[91879.015040] RSP: 0000:ffffbdb2a1437b30 EFLAGS: 00010246
[91879.015044] RAX: ffff970cc3dbdd30 RBX: 0000000000000001 RCX: 0000000000000000
[91879.015047] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
[91879.015048] RBP: fffff49809008300 R08: 0000000000000000 R09: 0000000000000000
[91879.015050] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000001
[91879.015052] R13: fffff49809008300 R14: ffff9711fe8d7bb0 R15: ffff971191976200
[91879.015054] FS:  00007fceec2f4740(0000) GS:ffff9714eae00000(0000) knlGS:0000000000000000
[91879.015056] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[91879.015058] CR2: 00007fceec576000 CR3: 000000053742a001 CR4: 00000000001706e0
[91879.015061] Call Trace:
[91879.015063]  <TASK>
[91879.015066]  insert_pfn+0x10e/0x160
[91879.015074]  __vm_insert_mixed+0xb0/0xd0
[91879.015079]  dax_fault_iter+0x742/0xa40
[91879.015088]  ? lock_is_held_type+0xd0/0x140
[91879.015101]  dax_iomap_pte_fault+0x1c9/0x640
[91879.015113]  __xfs_filemap_fault+0x305/0x410 [xfs]
[91879.015265]  __do_fault+0x36/0x1a0
[91879.015270]  __handle_mm_fault+0xc66/0x1470
[91879.015277]  handle_mm_fault+0x11a/0x3a0
[91879.015282]  do_user_addr_fault+0x1e0/0x6a0
[91879.015292]  exc_page_fault+0x77/0x2d0
[91879.015297]  asm_exc_page_fault+0x27/0x30
[91879.015304] RIP: 0033:0x558d1bd1488e
[91879.015330] Code: c0 0f 84 e1 00 00 00 48 8b 05 8e c2 02 00 48 2b 58 10 49 8d 14 1c 45 85 f6 75 55 4d 85 e4 0f 8e c7 fe ff ff 48 8b 00 44 89 ee <44> 88 2c 18 48 8d 43 01 49 83 fc 01 0f 8e af fe ff ff 48 8b 0d 59
[91879.015333] RSP: 002b:00007ffd5dc58990 EFLAGS: 00010206
[91879.015336] RAX: 00007fceec576000 RBX: 0000000000000000 RCX: 0000000000001000
[91879.015338] RDX: 0000000000001000 RSI: 0000000000000058 RDI: 0000000000000000
[91879.015340] RBP: 0000558d1c2e83e0 R08: 1999999999999999 R09: 0000000000000000
[91879.015342] R10: 00007fceec4a0ac0 R11: 00007fceec4a13c0 R12: 0000000000001000
[91879.015343] R13: 0000000000000058 R14: 0000000000000000 R15: 0000000000001000
[91879.015350]  </TASK>
[91879.015352] Modules linked in: scsi_debug nls_utf8 hfsplus hfs vfat fat isofs binfmt_misc tls dm_dust nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_rejec
t_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack iTCO_wdt nf_defrag_ipv6 intel_pmc_bxt nf_defrag_ipv4 intel_rapl_msr iTCO_vendor_support intel_rapl_common sb_edac x86_pkg_temp_thermal ip_set intel_powerclamp coretemp rfkill nf_tables nfnetlink qrtr kvm_intel kvm irqbypass rapl sunrpc intel_cstate lpc_ich hpilo ipmi_ssif pktcdvd intel_uncore dax_pmem acpi_ipmi ioatdma tg
3 dca ipmi_si acpi_power_meter fuse zram xfs nd_pmem nd_btt crct10dif_pclmul crc32_pclmul crc32c_intel hpsa nd_e820 libnvdimm ghash_clmulni_intel serio_raw mgag200 scsi_transport_sas hpwdt at
a_generic pata_acpi scsi_dh_rdac scsi_dh_emc scsi_dh_alua dm_multipath ipmi_devintf ipmi_msghandler
[91879.015432] ---[ end trace 0000000000000000 ]---
[91879.042514] RIP: 0010:page_table_check_set.part.0+0x89/0xe0
[91879.042522] Code: 75 64 44 89 c1 f0 0f c1 08 83 c1 01 83 f9 01 7e 04 84 db 75 67 48 83 c6 01 48 03 15 41 a4 e6 01 4c 39 e6 74 4f 48 85 d2 75 c2 <0f> 0b f7 c5 ff 0f 00 00 75 a4 48 8b 45 00 
a9 00 00 01 00 74 99 48
[91879.042524] RSP: 0000:ffffbdb2a1437b30 EFLAGS: 00010246
[91879.042527] RAX: ffff970cc3dbdd30 RBX: 0000000000000001 RCX: 0000000000000000
[91879.042528] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
[91879.042529] RBP: fffff49809008300 R08: 0000000000000000 R09: 0000000000000000
[91879.042531] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000001
[91879.042532] R13: fffff49809008300 R14: ffff9711fe8d7bb0 R15: ffff971191976200
[91879.042534] FS:  00007fceec2f4740(0000) GS:ffff9714eae00000(0000) knlGS:0000000000000000
[91879.042536] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[91879.042537] CR2: 00007fceec576000 CR3: 000000053742a001 CR4: 00000000001706e0
[91879.042539] note: xfs_io[48184] exited with preempt_count 1
[91882.423346] systemd[1]: Created slice system-dbus\x2d:1.2\x2dorg.freedesktop.problems.slice - Slice /system/dbus-:1.2-org.freedesktop.problems.
[91882.429366] systemd[1]: Started dbus-:1.2-org.freedesktop.problems@0.service
[92015.775618] systemd[1]: dbus-:1.2-org.freedesktop.problems@0.service: Deactivated successfully.
[92332.353087] systemd[1]: Starting dnf-makecache.service - dnf makecache...
[92347.887809] systemd[1]: dnf-makecache.service: Deactivated successfully.
[92347.934938] systemd[1]: Finished dnf-makecache.service - dnf makecache.
[92347.935805] systemd[1]: dnf-makecache.service: Consumed 6.327s CPU time.

[92397.345084] sysrq: Show Blocked State
[92397.365912] task:xfs_io          state:D stack:14528 pid:48187 ppid: 47965 flags:0x00004006
[92397.408219] Call Trace:
[92397.420926]  <TASK>
[92397.430899]  __schedule+0x492/0x1640
[92397.447320]  ? lock_acquire+0x26a/0x2d0
[92397.464582]  ? rcu_read_lock_sched_held+0x10/0x70
[92397.485899]  ? lock_release+0x215/0x460
[92397.503192]  schedule+0x4e/0xb0
[92397.517338]  rwsem_down_write_slowpath+0x35a/0x710
[92397.538897]  down_write+0xad/0x110
[92397.554207]  exit_mmap+0x46/0x1a0
[92397.569167]  ? uprobe_clear_state+0x25/0x120
[92397.588450]  ? __mutex_unlock_slowpath+0x2a/0x260
[92397.610111]  ? uprobe_clear_state+0x68/0x120
[92397.630130]  mmput+0x71/0x150
[92397.643507]  do_exit+0x324/0xc40
[92397.658063]  ? rcu_read_lock_sched_held+0x10/0x70
[92397.679239]  do_group_exit+0x33/0xb0
[92397.695338]  get_signal+0xbbc/0xbc0
[92397.711029]  arch_do_signal_or_restart+0x30/0x770
[92397.732360]  ? __schedule+0x49a/0x1640
[92397.749793]  ? lock_is_held_type+0xe8/0x140
[92397.768690]  exit_to_user_mode_prepare+0x172/0x270
[92397.790278]  syscall_exit_to_user_mode+0x16/0x50
[92397.811059]  do_syscall_64+0x67/0x80
[92397.827127]  ? sched_clock_cpu+0xb/0xb0
[92397.844438]  ? lock_release+0x14f/0x460
[92397.861976]  ? _raw_spin_unlock_irq+0x24/0x50
[92397.883664]  ? lock_is_held_type+0xe8/0x140
[92397.904426]  ? do_syscall_64+0x67/0x80
[92397.922912]  ? lockdep_hardirqs_on+0x7d/0x100
[92397.944397]  ? do_syscall_64+0x67/0x80
[92397.961255]  ? lockdep_hardirqs_on+0x7d/0x100
[92397.981189]  entry_SYSCALL_64_after_hwframe+0x46/0xb0
[92398.004672] RIP: 0033:0x7fceec3fc422
[92398.020394] RSP: 002b:00007fceec2f2e30 EFLAGS: 00000293 ORIG_RAX: 0000000000000022
[92398.054524] RAX: fffffffffffffdfe RBX: 00007fceec2f3640 RCX: 00007fceec3fc422
[92398.086955] RDX: 0000000000000002 RSI: 0000000000000000 RDI: 0000000000000000
[92398.119583] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007ffd5dc5891f
[92398.151800] R10: 0000000000000008 R11: 0000000000000293 R12: ffffffffffffff80
[92398.183959] R13: 0000000000000016 R14: 00007ffd5dc58820 R15: 00007fceebaf3000
[92398.216164]  </TASK>
Comment 1 Zorro Lang 2022-05-30 07:19:18 UTC
I just tried the lastest mainline upstream linux, it's 100% reproducible for me. And it's not only reproducible on xfs, ext4 with DAX can reproduce it too[1]. So it might be a common bug from mm?

[1]
[  291.434410] run fstests generic/623 at 2022-05-30 15:14:32
[  291.768382] systemd[1]: Started fstests-generic-623.scope - /usr/bin/bash -c test -w /proc/self/oom_score_adj && echo 250 > /proc/self/oom_score_adj; exec ./tests/generic/623.
[  294.406254] EXT4-fs (pmem0p2): mounted filesystem with ordered data mode. Quota mode: none.
[  294.442928] EXT4-fs (pmem0p2): shut down requested (1)
[  294.443089] Aborting journal on device pmem0p2-8.
[  294.477893] EXT4-fs (pmem0p2): unmounting filesystem.
[  294.479943] systemd[1]: mnt-scratch.mount: Deactivated successfully.
[  294.960374] EXT4-fs (pmem0p2): mounted filesystem with ordered data mode. Quota mode: none.
[  295.043716] ------------[ cut here ]------------
[  295.043722] kernel BUG at mm/page_table_check.c:51!
[  295.043736] invalid opcode: 0000 [#1] PREEMPT SMP KASAN PTI
[  295.112220] CPU: 23 PID: 1936 Comm: xfs_io Tainted: G S        I       5.18.0-mainline+ #2
[  295.149570] Hardware name: HP ProLiant DL380p Gen8, BIOS P70 08/02/2014
[  295.180088] RIP: 0010:page_table_check_set.part.0+0x184/0x300
[  295.208336] Code: 48 8b 04 24 80 38 00 0f 85 2b 01 00 00 48 83 c3 01 4c 03 3d 5e e9 5a 03 4c 39 eb 0f 84 05 01 00 00 4d 85 ff 0f 85 53 ff ff ff <0f> 0b f7 c3 ff 0f 00 00 0f 85 f5 fe ff ff 
be 08 00 00 00 48 89 df
[  295.293142] RSP: 0000:ffffc9000d50f708 EFLAGS: 00010246
[  295.316834] RAX: dffffc0000000000 RBX: ffffea00092180c0 RCX: ffffffff85a98a25
[  295.349059] RDX: 1ffffd400124301b RSI: 0000000000000001 RDI: ffffea00092180d8
[  295.381191] RBP: ffffea00092180c0 R08: 0000000000000000 R09: ffff88801bfa7607
[  295.413394] R10: ffffed10037f4ec0 R11: 0000000000000005 R12: 0000000000000000
[  295.445718] R13: 0000000000000001 R14: ffff88801bfa7600 R15: 0000000000000000
[  295.477844] FS:  00007f590a83f740(0000) GS:ffff888085000000(0000) knlGS:0000000000000000
[  295.518073] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  295.547270] CR2: 00007f590aac1000 CR3: 0000000049404003 CR4: 00000000001706e0
[  295.581391] Call Trace:
[  295.602075]  __page_table_check_pte_set+0x28f/0x350
[  295.624144]  ? __page_table_check_pte_clear+0x2b0/0x2b0
[  295.647669]  ? percpu_ref_put_many.constprop.0+0x1a0/0x1a0
[  295.672391]  ? __get_locked_pte+0x1b8/0x2e0
[  295.691432]  insert_pfn+0x22d/0x340
[  295.707093]  ? vm_map_pages_zero+0x10/0x10
[  295.725738]  ? pfn_modify_allowed+0x64/0x2b0
[  295.745392]  ? track_pfn_remap+0x1d0/0x1d0
[  295.764201]  __vm_insert_mixed+0x17f/0x1e0
[  295.782965]  ? vmf_insert_pfn+0x60/0x60
[  295.800382]  ? __dax_invalidate_entry+0x1c0/0x1c0
[  295.822137]  ? dax_direct_access+0x111/0x1b0
[  295.841356]  dax_fault_iter+0x100f/0x1bf0
[  295.859436]  ? grab_mapping_entry+0x4e0/0x4e0
[  295.879076]  ? iomap_iter+0xa02/0x10a0
[  295.895964]  dax_iomap_pte_fault+0x3f4/0xb80
[  295.915252]  ? dax_writeback_mapping_range+0xeb0/0xeb0
[  295.938698]  ? __ext4_journal_start_sb+0x345/0x460
[  295.960264]  ext4_dax_huge_fault+0x44f/0x940
[  295.979436]  ? ext4_file_open+0xa50/0xa50
[  295.998477]  ? var_wake_function+0x260/0x260
[  296.020138]  __do_fault+0xf8/0x4e0
[  296.037077]  do_fault+0x778/0xee0
[  296.053492]  ? restore_exclusive_pte+0x5d0/0x5d0
[  296.076166]  __handle_mm_fault+0xf82/0x26d0
[  296.095032]  ? vm_iomap_memory+0x140/0x140
[  296.113504]  ? count_memcg_events.constprop.0+0x40/0x50
[  296.137317]  handle_mm_fault+0x20e/0x750
[  296.154958]  do_user_addr_fault+0x345/0xd70
[  296.173803]  ? rcu_read_lock_sched_held+0x3c/0x70
[  296.194972]  exc_page_fault+0x65/0x100
[  296.213635]  asm_exc_page_fault+0x27/0x30
[  296.232854] RIP: 0033:0x55d4406da88e
[  296.249860] Code: c0 0f 84 e1 00 00 00 48 8b 05 8e c2 02 00 48 2b 58 10 49 8d 14 1c 45 85 f6 75 55 4d 85 e4 0f 8e c7 fe ff ff 48 8b 00 44 89 ee <44> 88 2c 18 48 8d 43 01 49 83 fc 01 0f 8e 
af fe ff ff 48 8b 0d 59
[  296.334980] RSP: 002b:00007fff29d58370 EFLAGS: 00010206
[  296.358575] RAX: 00007f590aac1000 RBX: 0000000000000000 RCX: 0000000000001000
[  296.390927] RDX: 0000000000001000 RSI: 0000000000000058 RDI: 0000000000000000
[  296.423124] RBP: 000055d441b093e0 R08: 1999999999999999 R09: 0000000000000000
[  296.455428] R10: 00007f590a9ebac0 R11: 00007f590a9ec3c0 R12: 0000000000001000
[  296.489563] R13: 0000000000000058 R14: 0000000000000000 R15: 0000000000000200
[  296.524615]  </TASK>
[  296.535495] Modules linked in: nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ip
v6 iTCO_wdt intel_rapl_msr intel_pmc_bxt nf_defrag_ipv4 iTCO_vendor_support intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp ip_set coretemp kvm_intel kvm irqbypass nf_tables rfkill rapl intel_cstate nfnetlink qrtr intel_uncore dax_pmem pcspkr sunrpc lpc_ich hpilo pktcdvd ipmi_ssif acpi_ipmi tg3 ioatdma ipmi_si acpi_power_meter dca fuse zram xfs nd_pmem nd_btt crct
10dif_pclmul crc32_pclmul nd_e820 crc32c_intel libnvdimm ghash_clmulni_intel hpsa serio_raw mgag200 hpwdt scsi_transport_sas ata_generic pata_acpi scsi_dh_rdac scsi_dh_emc scsi_dh_alua dm_mul
tipath ipmi_devintf ipmi_msghandler
[  296.859756] ---[ end trace 0000000000000000 ]---
[  296.897217] RIP: 0010:page_table_check_set.part.0+0x184/0x300
[  296.897228] Code: 48 8b 04 24 80 38 00 0f 85 2b 01 00 00 48 83 c3 01 4c 03 3d 5e e9 5a 03 4c 39 eb 0f 84 05 01 00 00 4d 85 ff 0f 85 53 ff ff ff <0f> 0b f7 c3 ff 0f 00 00 0f 85 f5 fe ff ff 
be 08 00 00 00 48 89 df
[  296.897233] RSP: 0000:ffffc9000d50f708 EFLAGS: 00010246
[  296.897238] RAX: dffffc0000000000 RBX: ffffea00092180c0 RCX: ffffffff85a98a25
[  296.897241] RDX: 1ffffd400124301b RSI: 0000000000000001 RDI: ffffea00092180d8
[  296.897244] RBP: ffffea00092180c0 R08: 0000000000000000 R09: ffff88801bfa7607
[  296.897247] R10: ffffed10037f4ec0 R11: 0000000000000005 R12: 0000000000000000
[  296.897249] R13: 0000000000000001 R14: ffff88801bfa7600 R15: 0000000000000000
[  296.897253] FS:  00007f590a83f740(0000) GS:ffff888085000000(0000) knlGS:0000000000000000
[  296.897256] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  296.897260] CR2: 00007f590aac1000 CR3: 0000000049404003 CR4: 00000000001706e0
[  296.897264] note: xfs_io[1936] exited with preempt_count 1
Comment 2 Zorro Lang 2022-05-30 15:18:38 UTC
Created attachment 301076 [details]
kernel config file
Comment 3 Zorro Lang 2022-05-30 18:28:57 UTC
Created attachment 301077 [details]
unreproducible kernel config file

Note You need to log in before you can comment on or make changes to this bug.