Bug 215702 - [emmc] mmc0: Timeout waiting for hardware interrupt.
Summary: [emmc] mmc0: Timeout waiting for hardware interrupt.
Status: NEW
Alias: None
Product: Drivers
Classification: Unclassified
Component: MMC/SD (show other bugs)
Hardware: x86-64 Linux
: P1 high
Assignee: drivers_mmc-sd
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-19 05:47 UTC by Jack Xing
Modified: 2022-03-19 05:47 UTC (History)
0 users

See Also:
Kernel Version: 3.10.0-514.26.2.el7.x86_64
Subsystem:
Regression: No
Bisected commit-id:


Attachments
system log about emmc issue (24.72 KB, text/plain)
2022-03-19 05:47 UTC, Jack Xing
Details

Description Jack Xing 2022-03-19 05:47:49 UTC
Created attachment 300582 [details]
system log about emmc issue

We have encountered the following problems in the operation of our equipment.

---------------------------------
[2660450.914500] mmc0: Timeout waiting for hardware interrupt.
[2660450.918756] mmcblk0: error -110 sending stop command, original cmd response 0x900, card status 0x400e00
[2660524.052290] <LWLog:Info:> [LinkNotify] Type=33[Link Down] Subype=0 Id=533109 Value=0 SessionId=0
[2660525.067924] <LWLog:Info:> [LinkNotify] Type=32[Link UP] Subype=0 Id=533109 Value=104 SessionId=0
[2660650.961923] INFO: task xfsaild/dm-0:420 blocked for more than 120 seconds.
[2660650.961927] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[2660650.961930] xfsaild/dm-0    D ffff880175516000     0   420      2 0x00000000
[2660650.961935]  ffff880035c13d58 0000000000000046 ffff880035d48fb0 ffff880035c13fd8
[2660650.961939]  ffff880035c13fd8 ffff880035c13fd8 ffff880035d48fb0 ffff8801756b0e00
[2660650.961942]  0000000000000000 ffff880035d48fb0 ffff880035d57128 ffff880175516000
[2660650.961945] Call Trace:
[2660650.961955]  [<ffffffff8168c7f9>] schedule+0x29/0x70
[2660650.961999]  [<ffffffffa02523ed>] _xfs_log_force+0x1bd/0x2b0 [xfs]
[2660650.962023]  [<ffffffff810c54e0>] ? wake_up_state+0x20/0x20
[2660650.962049]  [<ffffffffa0252506>] xfs_log_force+0x26/0x80 [xfs]
[2660650.962077]  [<ffffffffa025ddd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[2660650.962113]  [<ffffffffa025df2a>] xfsaild+0x15a/0x660 [xfs]
[2660650.962139]  [<ffffffffa025ddd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[2660650.962143]  [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[2660650.962146]  [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[2660650.962150]  [<ffffffff81697758>] ret_from_fork+0x58/0x90
[2660650.962153]  [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[2660650.962158] INFO: task auditd:674 blocked for more than 120 seconds.
[2660650.962160] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[2660650.962161] auditd          D ffffffff8168a850     0   674      1 0x00000000
[2660650.962165]  ffff88000001fc00 0000000000000086 ffff8801755cbec0 ffff88000001ffd8
[2660650.962168]  ffff88000001ffd8 ffff88000001ffd8 ffff8801755cbec0 ffff88017fc96c40
[2660650.962171]  0000000000000000 7fffffffffffffff ffff88017ff985e8 ffffffff8168a850
[2660650.962174] Call Trace:
[2660650.962178]  [<ffffffff8168a850>] ? bit_wait+0x50/0x50
[2660650.962181]  [<ffffffff8168c7f9>] schedule+0x29/0x70
[2660650.962184]  [<ffffffff8168a239>] schedule_timeout+0x239/0x2c0
[2660650.962188]  [<ffffffff8130d9ab>] ? deadline_add_request+0x5b/0x70
[2660650.962193]  [<ffffffff810eb0dc>] ? ktime_get_ts64+0x4c/0xf0
[2660650.962196]  [<ffffffff8168a850>] ? bit_wait+0x50/0x50
[2660650.962199]  [<ffffffff8168bd9e>] io_schedule_timeout+0xae/0x130
[2660650.962202]  [<ffffffff8168be38>] io_schedule+0x18/0x20
[2660650.962205]  [<ffffffff8168a861>] bit_wait_io+0x11/0x50
[2660650.962208]  [<ffffffff8168a385>] __wait_on_bit+0x65/0x90
[2660650.962212]  [<ffffffff81180221>] wait_on_page_bit+0x81/0xa0
[2660650.962215]  [<ffffffff810b1be0>] ? wake_bit_function+0x40/0x40
[2660650.962218]  [<ffffffff81180351>] __filemap_fdatawait_range+0x111/0x190
[2660650.962222]  [<ffffffff8118d94e>] ? do_writepages+0x1e/0x40
[2660650.962225]  [<ffffffff811803e4>] filemap_fdatawait_range+0x14/0x30
[2660650.962228]  [<ffffffff81182666>] filemap_write_and_wait_range+0x56/0x90
[2660650.962253]  [<ffffffffa02332b6>] xfs_file_fsync+0x66/0x1e0 [xfs]
[2660650.962258]  [<ffffffff81230695>] do_fsync+0x65/0xa0
[2660650.962261]  [<ffffffff81230960>] SyS_fsync+0x10/0x20
[2660650.962264]  [<ffffffff81697809>] system_call_fastpath+0x16/0x1b


[2661050.917711] mmc0: Card stuck in programming state! mmcblk0 card_busy_detect
[2661050.917727] blk_update_request: I/O error, dev mmcblk0, sector 35661352
[2661050.917731] blk_update_request: I/O error, dev mmcblk0, sector 35661360
[2661050.917734] blk_update_request: I/O error, dev mmcblk0, sector 35661368
[2661050.917736] blk_update_request: I/O error, dev mmcblk0, sector 35661376
[2661050.917739] blk_update_request: I/O error, dev mmcblk0, sector 35661384
[2661050.917742] blk_update_request: I/O error, dev mmcblk0, sector 35661392
[2661050.917744] blk_update_request: I/O error, dev mmcblk0, sector 35661400
[2661050.917747] blk_update_request: I/O error, dev mmcblk0, sector 35661408
[2661050.917750] blk_update_request: I/O error, dev mmcblk0, sector 35661416
[2661050.917752] blk_update_request: I/O error, dev mmcblk0, sector 35661424
[2661050.917827] Buffer I/O error on dev dm-0, logical block 3431109, lost async page write
[2661050.917836] Buffer I/O error on dev dm-0, logical block 3431110, lost async page write
[2661050.917839] Buffer I/O error on dev dm-0, logical block 3431111, lost async page write
[2661050.917841] Buffer I/O error on dev dm-0, logical block 3431112, lost async page write
[2661050.917844] Buffer I/O error on dev dm-0, logical block 3431113, lost async page write
[2661050.917846] Buffer I/O error on dev dm-0, logical block 3431114, lost async page write
[2661050.917848] Buffer I/O error on dev dm-0, logical block 3431115, lost async page write
[2661050.917851] Buffer I/O error on dev dm-0, logical block 3431116, lost async page write
[2661050.917853] Buffer I/O error on dev dm-0, logical block 3431117, lost async page write
[2661050.917855] Buffer I/O error on dev dm-0, logical block 3431118, lost async page write
[2661050.926374] mmc0: cache flush error -110
[2661050.934646] mmc0: cache flush error -110
[2661060.961667] mmc0: Timeout waiting for hardware interrupt.
[2661060.963680] ------------[ cut here ]------------
[2661060.963697] WARNING: at drivers/mmc/host/sdhci.c:1004 sdhci_send_command+0x355/0x3b0 [sdhci]()
[2661060.963699] Modules linked in: lwconn(OE) sm4(OE) igb(OE) ixgbe(OE) bridge stp llc sha512_ssse3 sha512_generic drbg ansi_cprng intel_powerclamp coretemp iosf_mbi kvm_intel kvm iTCO_wdt iTCO_vendor_support irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw ipmi_ssif gf128mul glue_helper qat_c3xxx ablk_helper i2c_i801 cryptd pcspkr intel_qat authenc i2c_algo_bit ipmi_msghandler i2c_ismt shpchp i2c_core ip_tables xfs libcrc32c mmc_block sdhci_pci crct10dif_pclmul crct10dif_common dca sdhci ptp mmc_core crc32c_intel pps_core ahci libahci libata fjes dm_mirror dm_region_hash dm_log dm_mod cdc_ether(OE) GobiNet(OE) qmi_wwan_q(OE) cdc_wdm usbnet(OE) mii option(OE) usb_wwan(OE) nf_conntrack [last unloaded: igb]
[2661060.963743] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G        W  OE  ------------   3.10.0-514.26.2.el7.x86_64 #1
[2661060.963745] Hardware name: Default string Default string/LW3002-XH, BIOS 5.13 (G168-027) 11/16/2020
[2661060.963747]  0000000000000000 cd6e6ca033004373 ffff88017fd03d28 ffffffff81687133
[2661060.963751]  ffff88017fd03d60 ffffffff81085cb0 ffff8800365da500 ffff8800365da500
[2661060.963754]  0000000000000003 ffff880175525930 ffff8800365da500 ffff88017fd03d70
[2661060.963757] Call Trace:
[2661060.963759]  <IRQ>  [<ffffffff81687133>] dump_stack+0x19/0x1b
[2661060.963770]  [<ffffffff81085cb0>] warn_slowpath_common+0x70/0xb0
[2661060.963773]  [<ffffffff81085dfa>] warn_slowpath_null+0x1a/0x20
[2661060.963778]  [<ffffffffa015ff45>] sdhci_send_command+0x355/0x3b0 [sdhci]
[2661060.963783]  [<ffffffffa01600b0>] sdhci_finish_data+0x110/0x370 [sdhci]
[2661060.963788]  [<ffffffffa0160310>] ? sdhci_finish_data+0x370/0x370 [sdhci]
[2661060.963793]  [<ffffffffa016038d>] sdhci_timeout_timer+0x7d/0xc0 [sdhci]
[2661060.963796]  [<ffffffff81095eb6>] call_timer_fn+0x36/0x110
[2661060.963801]  [<ffffffffa0160310>] ? sdhci_finish_data+0x370/0x370 [sdhci]
[2661060.963804]  [<ffffffff81098ba7>] run_timer_softirq+0x237/0x340
[2661060.963808]  [<ffffffff8108f63f>] __do_softirq+0xef/0x280
[2661060.963812]  [<ffffffff8169929c>] call_softirq+0x1c/0x30
[2661060.963816]  [<ffffffff8102d365>] do_softirq+0x65/0xa0
[2661060.963819]  [<ffffffff8108f9d5>] irq_exit+0x115/0x120
[2661060.963822]  [<ffffffff81699f15>] smp_apic_timer_interrupt+0x45/0x60
[2661060.963824]  [<ffffffff8169845d>] apic_timer_interrupt+0x6d/0x80
[2661060.963826]  <EOI>  [<ffffffff81514eb2>] ? cpuidle_enter_state+0x52/0xc0
[2661060.963832]  [<ffffffff81514ff9>] cpuidle_idle_call+0xd9/0x210
[2661060.963836]  [<ffffffff8103512e>] arch_cpu_idle+0xe/0x30
[2661060.963840]  [<ffffffff810e82f5>] cpu_startup_entry+0x245/0x290
[2661060.963844]  [<ffffffff8104f0da>] start_secondary+0x1ba/0x230
[2661060.963847] ---[ end trace 42ca8c72f9e7a97b ]---
[2661060.966757] mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400e00
[2661309.490931] <LWLog:Info:> [LinkNotify] Type=33[Link Down] Subype=0 Id=533109 Value=0 SessionId=0
[2661310.515420] <LWLog:Info:> [LinkNotify] Type=32[Link UP] Subype=0 Id=533109 Value=105 SessionId=0
[2661660.965885] mmc0: Card stuck in programming state! mmcblk0 card_busy_detect
[2661660.965899] blk_update_request: 84 callbacks suppressed
[2661660.965902] blk_update_request: I/O error, dev mmcblk0, sector 34660233
[2661660.965904] blk_update_request: I/O error, dev mmcblk0, sector 34660241
[2661660.965906] blk_update_request: I/O error, dev mmcblk0, sector 34660249
[2661660.965939] XFS (dm-0): metadata I/O error: block 0x1938f89 ("xlog_iodone") error 5 numblks 64
[2661660.974849] XFS (dm-0): xfs_do_force_shutdown(0x2) called from line 1203 of file fs/xfs/xfs_log.c.  Return address = 0xffffffffa0250590
[2661660.975005] XFS (dm-0): Log I/O Error Detected.  Shutting down filesystem
[2661660.975086] XFS (dm-0): xfs_log_force: error -5 returned.
[2661660.975124] XFS (dm-0): xfs_log_force: error -5 returned.
[2661660.982191] XFS (dm-0): Please umount the filesystem and rectify the problem(s)
[2661661.025893] XFS (dm-0): xfs_log_force: error -5 returned.

-------------------------

Note You need to log in before you can comment on or make changes to this bug.