Bug 194967

Summary:	database server hangs
Product:	IO/Storage	Reporter:	Tomas Mozes (hydrapolic)
Component:	Block Layer	Assignee:	other_other
Status:	RESOLVED OBSOLETE
Severity:	normal	CC:	bugzilla.kernel.org
Priority:	P1
Hardware:	x86-64
OS:	Linux
Kernel Version:	4.4, 4.9	Subsystem:
Regression:	No	Bisected commit-id:
Attachments:	sysrq-l.log sysrq-m.log sysrq-w.log sysrq-w.log (case 2)

Description Tomas Mozes 2017-03-23 13:17:12 UTC

Hello,
we are observing system hangs after upgrading kernel from 4.1 to 4.4. This happens on a database server running MariaDB, hot backuping with Percona Xtrabackup and when doing rsync (mostly deletion on the server). Happens on three different servers, on both ext4 and xfs. Can be reproduced on kernel 4.4.55, 4.9.16, does not happen with kernel 4.1.39. This is a virtual machine (xen-domU) having disks on lvm.

There is nothing in dmesg, but server load goes up, cannot even do `lsof` or `cat` files. A hard system reboot of the virtual machine puts it back to normal.

https://bugs.launchpad.net/percona-xtrabackup/+bug/1625984
https://jira.mariadb.org/browse/MDEV-10852

Comment 1 Tomas Mozes 2017-03-23 13:17:57 UTC

Created attachment 255453 [details]
sysrq-l.log

echo l > /proc/sysrq-trigger

Comment 2 Tomas Mozes 2017-03-23 13:18:42 UTC

Created attachment 255455 [details]
sysrq-m.log

echo m > /proc/sysrq-trigger

Comment 3 Tomas Mozes 2017-03-23 13:19:26 UTC

Created attachment 255457 [details]
sysrq-w.log

echo w > /proc/sysrq-trigger

Comment 4 Tomas Mozes 2017-03-23 13:25:05 UTC

Sadly, this is not related to https://bugzilla.kernel.org/show_bug.cgi?id=119841

Comment 5 Tomas Mozes 2017-05-30 07:22:41 UTC

Created attachment 256765 [details]
sysrq-w.log (case 2)

sysrq-w.log (case 2)

Comment 6 Tomas Mozes 2017-05-30 07:23:59 UTC

Attached a new occurrence with kernel 4.4.67.

Comment 7 Tomas Mozes 2017-10-27 06:06:37 UTC

Cannot reproduce with kernel 4.12+. Maybe related to #193331.