OS: Arch Linux Linux 6.4.4 introduces 100% usage in one core of the CPU with io-wait operations when MariaDB 11.0.2 (mysqld process) is running, even if the database is completely empty, this problem still occurs. To prevent this bug for happening it is necessary to downgrade to Linux <6.4.3. Linux 6.4.4 without Arch Linux patches, Linux Mainline 6.5-rc1 and 6.5-rc2 don't fix the problem either. Strangely there is also no sign that the mysqld process is using an absurd amount of io-wait on the processor core, it was found by purely discarding and killing processes until the processor core usage was normal and without io-wait operations. No tool to monitor I/O operations to find was able to find that mysqld was the cause of this bug (not even top). I also used different commits prior to the 6.4.4 release using `git bisect` and none of them introduced this bug. There is no journal logs nor dmesg logs related to this bug. It independent of the file system used (I guess, it happens on EXT4 and BTRFS, but it doesn't seem to be related with the file system either) How to reproduce it?: - Install and use the Linux 6.4.4 Release - Install MariaDB, initialize a empty database and run the process with your preferred init system - Use htop or top to see the high io-wait usage in one core of the CPU Arch Linux forums thread (Where we tested different kernel commits and there is other people with the same problem): https://bbs.archlinux.org/viewtopic.php?id=287343 If i need to provide more information, just ask for it... (This bug was found originally because i use Akonadi, a PIM layer made by KDE Plasma developers that uses MariaDB/MySQL to store data)
I have to note that i just tested Linux LTS 6.1.39
Sorry, ignore the the previous message... I just tested Linux LTS 6.1.39 and it also introduces the same io-wait bug but if i downgrade back to Linux LTS 6.1.38, the bug doesn't exist and it works as intended.
Reported as https://lore.kernel.org/lkml/12251678.O9o76ZdvQC@natalenko.name/
I confirm the same issue is in linus' tree as of git HEAD at commit c2782531397f5cb19ca3f8f9c17727f1cdf5bee8. The Arch forum suggests 6.44 stable is fixed by reverting commit bd4f737b145d85c7183ec879ce46b57ce64362e1 The corresponding commit in mainline is 8a796565cec3601071cbbd27d6304e202019d014
I confirm this is resolved for me running kernel 6.4.4 built with commit bd4f737b145d85c7183ec879ce46b57ce64362e1 reverted.
Would you be able to test the patch from here: [1]? [1] https://lore.kernel.org/lkml/11ded843-ac08-2306-ad0f-586978d038b1@kernel.dk/raw
tested and acked on lkml. I tested 6.4.6 from stable with Jens' patch applied all works fine now.
*** Bug 217700 has been marked as a duplicate of this bug. ***
Tested kernel 6.4.6 built with the referenced patch, and the anomalous 100% cpu activity on a single core is no longer displayed in the System Monitor widget, and mpstat no longer shows >94% io-wait. So this resolves the issue for my systems.
I tested https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=io_uring-6.5&id=7b72d661f1f2f950ab8c12de7e2bc48bdac8ed69 It works without problems, now there is no false io-wait reflected into the CPU.
Also, this is my first Bugzilla report, since it's fixed in the "Linux 4.x block layer trees" but not in mainline, I should mark this bug as RESOLVED?
I tested the fix on top of 6.5.0-rc3 and it helps. Is this going to be added to 6.5.0-rc4?
Looks like it landed in the mainline, so the fix should be included 6.5.0-rc4.
Fixed in 6.4.8. Thanks everyone for the help. Ref: https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/commit/queue-6.4/io_uring-gate-iowait-schedule-on-having-pending-requests.patch?id=2b8c242ac869eae3d96b712fdb9940e9cd1e0d69