Bug 15150 - xfslogd crashes
Summary: xfslogd crashes
Status: RESOLVED CODE_FIX
Alias: None
Product: File System
Classification: Unclassified
Component: XFS (show other bugs)
Hardware: i386 Linux
: P1 high
Assignee: XFS Guru
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-27 16:25 UTC by Abdelghani Ouchabane
Modified: 2010-02-01 15:44 UTC (History)
1 user (show)

See Also:
Kernel Version: kernel-2.6.27.25-78.2.56.fc9
Subsystem:
Regression: Yes
Bisected commit-id:


Attachments

Description Abdelghani Ouchabane 2010-01-27 16:25:45 UTC
Hallo,
  I have a system based on Fedora 9 ( kernel-2.6.27.25-78.2.56.fc9 ), my system is keeping to mount/umount a XFS partition every 5 seconds. After a while the system gets crash with the following logs :

Thanks in advance, if you need further information, I will do it with pleasure.

Cheers,
Abdelghani


BUG: unable to handle kernel NULL pointer dereference at 00000000
IP: [<c04158b7>] __wake_up_common+0x15/0x52
*pde = 318a7067 *pte = 00000000 
Oops: 0000 [#1] PREEMPT 
Modules linked in: ipv6 autofs4 ext3 jbd mbcache cgosdrv(P) i2c_i801 iTCO_wdt
iTCO_vendor_support mcs7830 usbnet mii pciehp snd_hda_intel snd_seq_dummy
snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss
snd_pcm snd_timer snd_page_alloc snd_hwdep snd soundcore i915 drm i2c_algo_bit
i2c_core uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]

Pid: 21, comm: xfslogd/0 Tainted: P          (2.6.27.25-112.delos.current.i686
#1) 
EIP: 0060:[<c04158b7>] EFLAGS: 00010086 CPU: 0
EIP is at __wake_up_common+0x15/0x52
EAX: 00000000 EBX: fffffff4 ECX: 00000000 EDX: 00000003
ESI: 00000000 EDI: 00000000 EBP: 00000000 ESP: f6c7bf30
 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
Process xfslogd/0 (pid: 21, ti=f6c7b000 task=f72df400 task.ti=f6c7b000)
Stack: 00000003 f3775b84 f6c7b000 00000000 00000286 00000000 c0417a4f 00000000 
       00000000 00000001 00000000 f32f3c34 c04eef83 00000000 f32f3c00 00000000 
       f3775b40 f32f3300 f32f3300 00000000 00000000 00000000 00000002 f3775e9c 
Call Trace:
 [<c0417a4f>] __wake_up+0x1d/0x45
 [<c04eef83>] xlog_state_do_callback+0x2b1/0x2cc
 [<c0503d0e>] xfs_buf_iodone_work+0x0/0x4b
 [<c0503d4a>] xfs_buf_iodone_work+0x3c/0x4b
 [<c04275a5>] run_workqueue+0x8a/0x12a
 [<c04279c3>] worker_thread+0x0/0xc8
 [<c0427a80>] worker_thread+0xbd/0xc8
 [<c042a292>] autoremove_wake_function+0x0/0x2d
 [<c04279c3>] worker_thread+0x0/0xc8
 [<c0429e98>] kthread+0x36/0x5a
 [<c0429e62>] kthread+0x0/0x5a
 [<c0403cc3>] kernel_thread_helper+0x7/0x10
 =======================
Code: 0c 01 1d 3c 81 7c c0 11 15 40 81 7c c0 5b c3 8b 80 74 01 00 00 c3 55 57
89 cf 56 53 83 ec 08 89 44 24 04 89 14 24 8b 00 8d 58 f4 <8b> 43 0c eb 25 8b 2b
89 d8 ff 74 24 20 8b 54 24 04 8b 4c 24 20 
EIP: [<c04158b7>] __wake_up_common+0x15/0x52 SS:ESP 0068:f6c7bf30
---[ end trace b38fbaf4088c3d26 ]---
note: xfslogd/0[21] exited with preempt_count 1



Steps to reproduce:


System information:


[root@localhost ~]# cat /proc/version
Linux version 2.6.27.25-113.delos.current.i686.debug (abdelghani@alder.ezono.net) (gcc version 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) ) #1 PREEMPT Tue Jan 26 14:04:54 CET 2010



[root@localhost ~]# cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 14
model name	: Intel(R) Celeron(R) M CPU        423  @ 1.06GHz
stepping	: 12
cpu MHz		: 1063.991
cache size	: 1024 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss tm pbe nx constant_tsc arch_perfmon bts pni monitor tm2 xtpr
bogomips	: 2127.98
clflush size	: 64
power management:



[root@localhost ~]# cat /proc/modules
ipv6 230948 12 - Live 0xf9d47000
autofs4 17604 1 - Live 0xf9b53000
ext3 104584 1 - Live 0xf9b65000
jbd 38932 1 ext3, Live 0xf9b5a000
mbcache 6528 1 ext3, Live 0xf8afa000
dm9601 6400 0 - Live 0xf89b8000
usbnet 12552 1 dm9601, Live 0xf8aba000
mii 4032 2 dm9601,usbnet, Live 0xf89d5000
cgosdrv 18520 2 - Live 0xf8ad1000 (P)
i2c_i801 7760 0 - Live 0xf88fd000
iTCO_wdt 9696 1 - Live 0xf89fb000
iTCO_vendor_support 2756 1 iTCO_wdt, Live 0xf89b4000
ehci_hcd 29196 0 - Live 0xf8ac8000
uhci_hcd 19024 0 - Live 0xf89c9000
pciehp 27864 0 - Live 0xf8ac0000
snd_hda_intel 67088 3 - Live 0xf8a81000
snd_seq_dummy 2436 0 - Live 0xf88fb000
snd_seq_oss 24724 0 - Live 0xf89e2000
snd_seq_midi_event 5568 1 snd_seq_oss, Live 0xf88d7000
snd_seq 39992 5 snd_seq_dummy,snd_seq_oss,snd_seq_midi_event, Live 0xf89d7000
snd_seq_device 5516 3 snd_seq_dummy,snd_seq_oss,snd_seq, Live 0xf88d4000
snd_pcm_oss 30144 0 - Live 0xf89ab000
snd_mixer_oss 12096 1 snd_pcm_oss, Live 0xf88c9000
snd_pcm 49480 3 snd_hda_intel,snd_pcm_oss, Live 0xf89bb000
snd_timer 16900 3 snd_seq,snd_pcm, Live 0xf89a5000
snd_page_alloc 6472 2 snd_hda_intel,snd_pcm, Live 0xf88cd000
snd_hwdep 6020 1 snd_hda_intel, Live 0xf886c000
snd 41716 13 snd_hda_intel,snd_seq_oss,snd_seq,snd_seq_device,snd_pcm_oss,snd_mixer_oss,snd_pcm,snd_timer,snd_hwdep, Live 0xf88ef000
soundcore 5444 1 snd, Live 0xf88c6000
i915 81732 7 - Live 0xf88da000
drm 141912 8 i915, Live 0xf8981000
i2c_algo_bit 4740 1 i915, Live 0xf886f000
i2c_core 17488 4 i2c_i801,i915,drm,i2c_algo_bit, Live 0xf8873000




[root@localhost ~]# cat /proc/ioports
0000-001f : dma1
0020-0021 : pic1
0040-0043 : timer0
0050-0053 : timer1
0060-0060 : keyboard
0064-0064 : keyboard
0070-0071 : rtc0
0080-008f : dma page reg
00a0-00a1 : pic2
00c0-00df : dma2
00f0-00ff : fpu
0170-0177 : 0000:00:1f.1
  0170-0177 : ata_piix
01f0-01f7 : 0000:00:1f.1
  01f0-01f7 : ata_piix
0376-0376 : 0000:00:1f.1
  0376-0376 : ata_piix
03c0-03df : vga+
03f6-03f6 : 0000:00:1f.1
  03f6-03f6 : ata_piix
0400-041f : 0000:00:1f.3
  0400-041f : i801_smbus
0480-04bf : 0000:00:1f.0
  0480-04bf : pnp 00:06
04d0-04d1 : pnp 00:06
0800-087f : 0000:00:1f.0
  0800-087f : pnp 00:06
    0800-0803 : ACPI PM1a_EVT_BLK
    0804-0805 : ACPI PM1a_CNT_BLK
    0808-080b : ACPI PM_TMR
    0810-0815 : ACPI CPU throttle
    0820-0820 : ACPI PM2_CNT_BLK
    0828-082f : ACPI GPE0_BLK
    0860-087f : iTCO_wdt
0cf8-0cff : PCI conf1
d480-d49f : 0000:00:1d.2
  d480-d49f : uhci_hcd
d800-d81f : 0000:00:1d.1
  d800-d81f : uhci_hcd
d880-d89f : 0000:00:1d.0
  d880-d89f : uhci_hcd
dc00-dc07 : 0000:00:02.0
e000-efff : PCI Bus 0000:03
  e800-e8ff : 0000:03:00.0
ffa0-ffaf : 0000:00:1f.1
  ffa0-ffaf : ata_piix




[root@localhost ~]# cat /proc/iomem
00000000-0009fbff : System RAM
0009fc00-0009ffff : reserved
000a0000-000bffff : Video RAM area
000c0000-000c7fff : Video ROM
000e0000-000fffff : reserved
  000f0000-000fffff : System ROM
00100000-3d5fffff : System RAM
  00400000-0069a821 : Kernel code
  0069a822-007972bf : Kernel data
  007e5000-00875bb3 : Kernel bss
3f7c0000-3f7cdfff : ACPI Tables
3f7ce000-3f7fffff : ACPI Non-volatile Storage
d0000000-dfffffff : 0000:00:02.0
e0000000-e3ffffff : PCI MMCONFIG 0
  e0000000-e3ffffff : pnp 00:0c
fdf00000-fdffffff : PCI Bus 0000:02
  fdf00000-fdf7ffff : 0000:02:00.0
  fdffe000-fdffefff : 0000:02:00.0
  fdfff000-fdffffff : 0000:02:00.0
fe980000-fe9fffff : 0000:00:02.1
fea37c00-fea37fff : 0000:00:1d.7
  fea37c00-fea37fff : ehci_hcd
fea38000-fea3bfff : 0000:00:1b.0
  fea38000-fea3bfff : ICH HD audio
fea40000-fea7ffff : 0000:00:02.0
fea80000-feafffff : 0000:00:02.0
feb00000-febfffff : PCI Bus 0000:03
  febc0000-febdffff : 0000:03:00.0
  febff000-febfffff : 0000:03:00.0
fec00000-fec00fff : pnp 00:0b
fed13000-fed19fff : pnp 00:01
fed1c000-fed1ffff : pnp 00:06
fed20000-fed3ffff : pnp 00:06
fed40000-fed8ffff : pnp 00:06
fee00000-fee00fff : reserved
ffb00000-ffffffff : reserved




[root@localhost ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: InnoDisk Corp. - Rev: 0712
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Comment 1 Christoph Hellwig 2010-01-27 16:40:49 UTC
On Wed, Jan 27, 2010 at 04:25:48PM +0000, bugzilla-daemon@bugzilla.kernel.org wrote:
> Hallo,
>   I have a system based on Fedora 9 ( kernel-2.6.27.25-78.2.56.fc9 ), my
>   system
> is keeping to mount/umount a XFS partition every 5 seconds. After a while the
> system gets crash with the following logs :
> 
> Thanks in advance, if you need further information, I will do it with
> pleasure.

Can you see if the patch below fixes it?


Index: linux-2.6/fs/xfs/xfs_log.c
===================================================================
--- linux-2.6.orig/fs/xfs/xfs_log.c	2009-11-09 22:09:08.858026060 +0100
+++ linux-2.6/fs/xfs/xfs_log.c	2009-11-09 22:13:13.958255857 +0100
@@ -1602,6 +1602,8 @@ xlog_dealloc_log(xlog_t *log)
 	xlog_in_core_t	*iclog, *next_iclog;
 	int		i;
 
+	xfs_flush_buftarg(log->l_mp->m_logdev_targp, 1);
+
 	iclog = log->l_iclog;
 	for (i=0; i<log->l_iclog_bufs; i++) {
 		sv_destroy(&iclog->ic_force_wait);
Comment 2 Abdelghani Ouchabane 2010-01-27 16:54:49 UTC
(In reply to comment #1)
> On Wed, Jan 27, 2010 at 04:25:48PM +0000, bugzilla-daemon@bugzilla.kernel.org
> wrote:
> > Hallo,
> >   I have a system based on Fedora 9 ( kernel-2.6.27.25-78.2.56.fc9 ), my
> system
> > is keeping to mount/umount a XFS partition every 5 seconds. After a while
> the
> > system gets crash with the following logs :
> > 
> > Thanks in advance, if you need further information, I will do it with
> pleasure.
> 
> Can you see if the patch below fixes it?
> 
> 
> Index: linux-2.6/fs/xfs/xfs_log.c
> ===================================================================
> --- linux-2.6.orig/fs/xfs/xfs_log.c    2009-11-09 22:09:08.858026060 +0100
> +++ linux-2.6/fs/xfs/xfs_log.c    2009-11-09 22:13:13.958255857 +0100
> @@ -1602,6 +1602,8 @@ xlog_dealloc_log(xlog_t *log)
>      xlog_in_core_t    *iclog, *next_iclog;
>      int        i;
> 
> +    xfs_flush_buftarg(log->l_mp->m_logdev_targp, 1);
> +
>      iclog = log->l_iclog;
>      for (i=0; i<log->l_iclog_bufs; i++) {
>          sv_destroy(&iclog->ic_force_wait);


Thanks a lot for your quick reply, I will try soon.
Comment 3 Abdelghani Ouchabane 2010-01-27 17:11:08 UTC
More information of my system :

[abdelghani@alder cyclades-ghani]$  /sandbox/abdelghani/rpmbuild/BUILD/kernel-2.6.27/linux-2.6.27.i686/scripts/ver_linux 
If some fields are empty or look unusual you may have an old version.
Compare to the current minimal requirements in Documentation/Changes.
 
Linux alder.ezono.net 2.6.27.7-53.fc9.i686 #1 SMP Thu Nov 27 02:29:03 EST 2008 i686 i686 i386 GNU/Linux
 
Gnu C                  4.3.0
Gnu make               3.81
binutils               2.18.50.0.6
util-linux             2.13.1
mount                  2.13.1
module-init-tools      3.4
e2fsprogs              1.41.3
pcmciautils            014
quota-tools            3.15.
Linux C Library        2.8
Dynamic linker (ldd)   2.8
Procps                 3.2.7
Net-tools              1.60
Kbd                    1.12
oprofile               0.9.3
Sh-utils               6.10
udev                   124
wireless-tools         29
Modules Loaded         usb_storage fuse vmnet ppdev parport_pc parport vmblock vsock vmci vmmon autofs4 nfs nfs_acl lockd sunrpc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 xt_state nf_conntrack xt_tcpudp ipt_REJECT iptable_filter ip_tables x_tables cpufreq_ondemand acpi_cpufreq ipv6 dm_multipath scsi_dh snd_hda_intel snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer r8169 i915 snd_page_alloc snd_hwdep mii snd drm dcdbas sr_mod e1000e soundcore joydev pcspkr serio_raw i2c_i801 cdrom i2c_algo_bit iTCO_wdt i2c_core iTCO_vendor_support sg floppy dm_snapshot dm_zero dm_mirror dm_log dm_mod ahci libata sd_mod scsi_mod crc_t10dif ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd
Comment 4 Abdelghani Ouchabane 2010-02-01 15:44:33 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > On Wed, Jan 27, 2010 at 04:25:48PM +0000,
> bugzilla-daemon@bugzilla.kernel.org
> > wrote:
> > > Hallo,
> > >   I have a system based on Fedora 9 ( kernel-2.6.27.25-78.2.56.fc9 ), my
> system
> > > is keeping to mount/umount a XFS partition every 5 seconds. After a while
> the
> > > system gets crash with the following logs :
> > > 
> > > Thanks in advance, if you need further information, I will do it with
> pleasure.
> > 
> > Can you see if the patch below fixes it?
> > 
> > 
> > Index: linux-2.6/fs/xfs/xfs_log.c
> > ===================================================================
> > --- linux-2.6.orig/fs/xfs/xfs_log.c    2009-11-09 22:09:08.858026060 +0100
> > +++ linux-2.6/fs/xfs/xfs_log.c    2009-11-09 22:13:13.958255857 +0100
> > @@ -1602,6 +1602,8 @@ xlog_dealloc_log(xlog_t *log)
> >      xlog_in_core_t    *iclog, *next_iclog;
> >      int        i;
> > 
> > +    xfs_flush_buftarg(log->l_mp->m_logdev_targp, 1);
> > +
> >      iclog = log->l_iclog;
> >      for (i=0; i<log->l_iclog_bufs; i++) {
> >          sv_destroy(&iclog->ic_force_wait);
> 
> 
> Thanks a lot for your quick reply, I will try soon.


Hallo Christoph,

  after testing my system for more than 72 hours, I can say that the bug was fixed by patching the kernel with your patch.


Thanks a lot .
Cheers,
Ghani

Note You need to log in before you can comment on or make changes to this bug.