Bug 12785 - kswapd block the whole system by IO blaster in some case
Summary: kswapd block the whole system by IO blaster in some case
Status: RESOLVED UNREPRODUCIBLE
Alias: None
Product: Memory Management
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: P1 low
Assignee: Andrew Morton
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-02-26 21:20 UTC by Martin Chen
Modified: 2009-08-12 03:18 UTC (History)
1 user (show)

See Also:
Kernel Version: 2.6.28.4
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Martin Chen 2009-02-26 21:20:45 UTC
Distribution:debian lenny with some experimental packages
Hardware Environment:ThinkPad SL 400 7DC with 2G memery
Software Environment:no swap partition,kernel with 4G memery support
Problem Description:
Some day, my box dived into a block while HDLED was blinking.
I switched to console from gdm, tried iotop by long waiting and found the killer was kswapd.
In "top" output, free memory is almost 50M.The most memory is cached by swap.
The system blocked even shutdown command wasn't effective.The box had been killed by pressing then power button.
BTW, there was no network available then, so there was no attack possibility.

I'd like to attach my kernel config file, but I don't know how to.For someone interesting, we may transfer the file my mail.crackevil@gmail.com

ps:these experimental packages installed

libdrm2_2.4.4+git+20090205+8b88036-1_i386.deb
libdrm-dev_2.4.4+git+20090205+8b88036-1_i386.deb
libdrm-intel1_2.4.4+git+20090205+8b88036-1_i386.deb
libdrm-nouveau1_2.4.4+git+20090205+8b88036-1_i386.deb
libgl1-mesa-dev_7.3-1_all.deb
libgl1-mesa-dri_7.3-1_i386.deb
libgl1-mesa-glx_7.3-1_i386.deb
libglu1-mesa_7.3-1_i386.deb
mesa-common-dev_7.3-1_all.deb
mesa-utils_7.3-1_i386.deb
xserver-common_2%3a1.5.99.902-1_all.deb
xserver-xorg-core_2%3a1.5.99.902-1_i386.deb
xkb-data_1.5-2_all.deb
Comment 1 Anonymous Emailer 2009-02-26 21:29:57 UTC
Reply-To: akpm@linux-foundation.org


(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

(uh-oh)

On Thu, 26 Feb 2009 21:20:46 -0800 (PST) bugme-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=12785
> 
>            Summary: kswapd block the whole system by IO blaster in some case
>            Product: Memory Management
>            Version: 2.5
>      KernelVersion: 2.6.28.4
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: low
>           Priority: P1
>          Component: Other
>         AssignedTo: akpm@osdl.org
>         ReportedBy: crackevil@gmail.com
> 
> 
> Distribution:debian lenny with some experimental packages
> Hardware Environment:ThinkPad SL 400 7DC with 2G memery
> Software Environment:no swap partition,kernel with 4G memery support
> Problem Description:
> Some day, my box dived into a block while HDLED was blinking.
> I switched to console from gdm, tried iotop by long waiting and found the
> killer was kswapd.
> In "top" output, free memory is almost 50M.The most memory is cached by swap.
> The system blocked even shutdown command wasn't effective.The box had been
> killed by pressing then power button.
> BTW, there was no network available then, so there was no attack possibility.
> 
> I'd like to attach my kernel config file, but I don't know how to.For someone
> interesting, we may transfer the file my mail.crackevil@gmail.com
> 
> ps:these experimental packages installed
> 
> libdrm2_2.4.4+git+20090205+8b88036-1_i386.deb
> libdrm-dev_2.4.4+git+20090205+8b88036-1_i386.deb
> libdrm-intel1_2.4.4+git+20090205+8b88036-1_i386.deb
> libdrm-nouveau1_2.4.4+git+20090205+8b88036-1_i386.deb
> libgl1-mesa-dev_7.3-1_all.deb
> libgl1-mesa-dri_7.3-1_i386.deb
> libgl1-mesa-glx_7.3-1_i386.deb
> libglu1-mesa_7.3-1_i386.deb
> mesa-common-dev_7.3-1_all.deb
> mesa-utils_7.3-1_i386.deb
> xserver-common_2%3a1.5.99.902-1_all.deb
> xserver-xorg-core_2%3a1.5.99.902-1_i386.deb
> xkb-data_1.5-2_all.deb
> 
> 
Comment 2 KOSAKI Motohiro 2009-02-26 21:48:47 UTC
> 
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> 
> (uh-oh)
> 
> On Thu, 26 Feb 2009 21:20:46 -0800 (PST) bugme-daemon@bugzilla.kernel.org
> wrote:
> 
> > http://bugzilla.kernel.org/show_bug.cgi?id=12785
> > 
> >            Summary: kswapd block the whole system by IO blaster in some
> case
> >            Product: Memory Management
> >            Version: 2.5
> >      KernelVersion: 2.6.28.4
> >           Platform: All
> >         OS/Version: Linux
> >               Tree: Mainline
> >             Status: NEW
> >           Severity: low
> >           Priority: P1
> >          Component: Other
> >         AssignedTo: akpm@osdl.org
> >         ReportedBy: crackevil@gmail.com
> > 
> > 
> > Distribution:debian lenny with some experimental packages

As far as I know, lenny (debian 5.0) use kernel 2.6.26.
then, recently changed code don't provide any hint.

Martin, if remove experimental package, do you still see the same issue?



> > Hardware Environment:ThinkPad SL 400 7DC with 2G memery
> > Software Environment:no swap partition,kernel with 4G memery support
> > Problem Description:
> > Some day, my box dived into a block while HDLED was blinking.
> > I switched to console from gdm, tried iotop by long waiting and found the
> > killer was kswapd.
> > In "top" output, free memory is almost 50M.The most memory is cached by
> swap.
> > The system blocked even shutdown command wasn't effective.The box had been
> > killed by pressing then power button.
> > BTW, there was no network available then, so there was no attack
> possibility.
> > 
> > I'd like to attach my kernel config file, but I don't know how to.For
> someone
> > interesting, we may transfer the file my mail.crackevil@gmail.com
> > 
> > ps:these experimental packages installed
> > 
> > libdrm2_2.4.4+git+20090205+8b88036-1_i386.deb
> > libdrm-dev_2.4.4+git+20090205+8b88036-1_i386.deb
> > libdrm-intel1_2.4.4+git+20090205+8b88036-1_i386.deb
> > libdrm-nouveau1_2.4.4+git+20090205+8b88036-1_i386.deb
> > libgl1-mesa-dev_7.3-1_all.deb
> > libgl1-mesa-dri_7.3-1_i386.deb
> > libgl1-mesa-glx_7.3-1_i386.deb
> > libglu1-mesa_7.3-1_i386.deb
> > mesa-common-dev_7.3-1_all.deb
> > mesa-utils_7.3-1_i386.deb
> > xserver-common_2%3a1.5.99.902-1_all.deb
> > xserver-xorg-core_2%3a1.5.99.902-1_i386.deb
> > xkb-data_1.5-2_all.deb
> > 
> > 
> 
Comment 3 Martin Chen 2009-02-26 22:00:38 UTC
Those experimental packages have been in use for 2 months, this issue
just occured and just once.
I think it's quite unexpectly random. Even in the same environment, I
can't make that issue alive again.

Today I'll look through the logs. Maybe in the syslog I can find
something valuable while other logs like dmesg, messages, kern.log,
etc have been found useless.

On Fri, Feb 27, 2009 at 1:48 PM, KOSAKI Motohiro
<kosaki.motohiro@jp.fujitsu.com> wrote:
>>
>> (switched to email.  Please respond via emailed reply-to-all, not via the
>> bugzilla web interface).
>>
>> (uh-oh)
>>
>> On Thu, 26 Feb 2009 21:20:46 -0800 (PST) bugme-daemon@bugzilla.kernel.org
>> wrote:
>>
>> > http://bugzilla.kernel.org/show_bug.cgi?id=12785
>> >
>> >            Summary: kswapd block the whole system by IO blaster in some
>> case
>> >            Product: Memory Management
>> >            Version: 2.5
>> >      KernelVersion: 2.6.28.4
>> >           Platform: All
>> >         OS/Version: Linux
>> >               Tree: Mainline
>> >             Status: NEW
>> >           Severity: low
>> >           Priority: P1
>> >          Component: Other
>> >         AssignedTo: akpm@osdl.org
>> >         ReportedBy: crackevil@gmail.com
>> >
>> >
>> > Distribution:debian lenny with some experimental packages
>
> As far as I know, lenny (debian 5.0) use kernel 2.6.26.
> then, recently changed code don't provide any hint.
>
> Martin, if remove experimental package, do you still see the same issue?
>
>
>
>> > Hardware Environment:ThinkPad SL 400 7DC with 2G memery
>> > Software Environment:no swap partition,kernel with 4G memery support
>> > Problem Description:
>> > Some day, my box dived into a block while HDLED was blinking.
>> > I switched to console from gdm, tried iotop by long waiting and found the
>> > killer was kswapd.
>> > In "top" output, free memory is almost 50M.The most memory is cached by
>> swap.
>> > The system blocked even shutdown command wasn't effective.The box had been
>> > killed by pressing then power button.
>> > BTW, there was no network available then, so there was no attack
>> possibility.
>> >
>> > I'd like to attach my kernel config file, but I don't know how to.For
>> someone
>> > interesting, we may transfer the file my mail.crackevil@gmail.com
>> >
>> > ps:these experimental packages installed
>> >
>> > libdrm2_2.4.4+git+20090205+8b88036-1_i386.deb
>> > libdrm-dev_2.4.4+git+20090205+8b88036-1_i386.deb
>> > libdrm-intel1_2.4.4+git+20090205+8b88036-1_i386.deb
>> > libdrm-nouveau1_2.4.4+git+20090205+8b88036-1_i386.deb
>> > libgl1-mesa-dev_7.3-1_all.deb
>> > libgl1-mesa-dri_7.3-1_i386.deb
>> > libgl1-mesa-glx_7.3-1_i386.deb
>> > libglu1-mesa_7.3-1_i386.deb
>> > mesa-common-dev_7.3-1_all.deb
>> > mesa-utils_7.3-1_i386.deb
>> > xserver-common_2%3a1.5.99.902-1_all.deb
>> > xserver-xorg-core_2%3a1.5.99.902-1_i386.deb
>> > xkb-data_1.5-2_all.deb
>> >
>> >
>>
>
>
>
>
Comment 4 KAMEZAWA Hiroyuki 2009-02-26 22:10:34 UTC
On Thu, 26 Feb 2009 21:29:18 -0800
Andrew Morton <akpm@linux-foundation.org> wrote:

> 
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> 
> (uh-oh)
> 
> On Thu, 26 Feb 2009 21:20:46 -0800 (PST) bugme-daemon@bugzilla.kernel.org
> wrote:
> 
> > http://bugzilla.kernel.org/show_bug.cgi?id=12785
> > 
> >            Summary: kswapd block the whole system by IO blaster in some
> case
> >            Product: Memory Management
> >            Version: 2.5
> >      KernelVersion: 2.6.28.4
> >           Platform: All
> >         OS/Version: Linux
> >               Tree: Mainline
> >             Status: NEW
> >           Severity: low
> >           Priority: P1
> >          Component: Other
> >         AssignedTo: akpm@osdl.org
> >         ReportedBy: crackevil@gmail.com
> > 
> > 
> > Distribution:debian lenny with some experimental packages
> > Hardware Environment:ThinkPad SL 400 7DC with 2G memery
> > Software Environment:no swap partition,kernel with 4G memery support
> > Problem Description:
> > Some day, my box dived into a block while HDLED was blinking.
> > I switched to console from gdm, tried iotop by long waiting and found the
> > killer was kswapd.
> > In "top" output, free memory is almost 50M.The most memory is cached by
> swap.
> > The system blocked even shutdown command wasn't effective.The box had been
> > killed by pressing then power button.

BTW, pages can be SwapCache even when there are no swap partition ?

Thanks,
-Kame
Comment 5 Roland Kletzing 2009-08-11 19:22:39 UTC
Martin, 

did that happen again ?

it`s a little bit weird how kwsapd can be an i/o killer when there is no swap attached. 

if this is not reproducible, then it`s almost impossible to debug this problem. 

holding this bugreport open is just a waste of time then...

@Kame:
>BTW, pages can be SwapCache even when there are no swap partition ?
i think that was just misinterpretation of top output, as "cached" appears in the line were swap information (total/used/free) is also displayed.

regards
roland
Comment 6 Martin Chen 2009-08-12 03:13:34 UTC
Roland,
when it happened, kswapd made my CPU too busy so that made a system dead. I
think memory in system maybe informative so I add some memory, so I didn't
mean it caused by no swap attached. actually I think on the spot, kswapd was
trapped into a dead loop :)-
since it never happened again, I'll close this bug report.
thank you

On Wed, Aug 12, 2009 at 3:22 AM, <bugzilla-daemon@bugzilla.kernel.org>wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=12785
>
>
> Roland Kletzing <devzero@web.de> changed:
>
>           What    |Removed                     |Added
>
> ----------------------------------------------------------------------------
>                 CC|                            |devzero@web.de
>
>
>
>
> --- Comment #5 from Roland Kletzing <devzero@web.de>  2009-08-11 19:22:39
> ---
> Martin,
>
> did that happen again ?
>
> it`s a little bit weird how kwsapd can be an i/o killer when there is no
> swap
> attached.
>
> if this is not reproducible, then it`s almost impossible to debug this
> problem.
>
> holding this bugreport open is just a waste of time then...
>
> @Kame:
> >BTW, pages can be SwapCache even when there are no swap partition ?
> i think that was just misinterpretation of top output, as "cached" appears
> in
> the line were swap information (total/used/free) is also displayed.
>
> regards
> roland
>
> --
> Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You reported the bug.
>

Note You need to log in before you can comment on or make changes to this bug.