Bug 10350 - __kfree_skb crashed the kernel?
Summary: __kfree_skb crashed the kernel?
Status: CLOSED OBSOLETE
Alias: None
Product: Process Management
Classification: Unclassified
Component: Preemption (show other bugs)
Hardware: All Linux
: P1 high
Assignee: networking_netfilter-iptables@kernel-bugs.osdl.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-03-28 12:17 UTC by Don Harter
Modified: 2012-05-18 10:43 UTC (History)
1 user (show)

See Also:
Kernel Version: 2.6.24.2-default #1 SMP
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Don Harter 2008-03-28 12:17:41 UTC
Latest working kernel version:
Earliest failing kernel version:
Distribution: Suse 10.3 untainted
Hardware Environment: AMD
Software Environment: KDE
Problem Description: kernel crashed when running ktorrent

Steps to reproduce: haven't yet
Mar 28 13:40:27 (none) syslog-ng[2578]: STATS: dropped 0
Mar 28 13:47:04 (none) kernel: general protection fault: 0000 [1] SMP
Mar 28 13:47:04 (none) kernel: CPU 1
Mar 28 13:47:04 (none) kernel: Modules linked in: af_packet snd_pcm_oss snd_mixer_oss snd_seq snd_seq_device ip6t_REJECT cpufreq_conservative cpufreq_ondemand cpufreq_userspace cpufreq_powersave powernow_k8 ip6table_mangle freq_table ip6table_filter ip6_tables ipv6 fuse dm_crypt loop dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd soundcore i2c_nforce2 button snd_page_alloc forcedeth i2c_core sg sd_mod edd fan generic thermal processor
Mar 28 13:47:04 (none) kernel: Pid: 5733, comm: ktorrent Not tainted 2.6.24.2-default #1
Mar 28 13:47:04 (none) kernel: RIP: 0010:[<ffffffff802866c1>]  [<ffffffff802866c1>] kfree+0x6c/0x9f
Mar 28 13:47:04 (none) kernel: RSP: 0018:ffff8100249c7ba8  EFLAGS: 00010082
Mar 28 13:47:04 (none) kernel: RAX: 0000000000000001 RBX: ffff810001000000 RCX: 0000000000000001
Mar 28 13:47:04 (none) kernel: RDX: ffff810001596e28 RSI: ffff810043e042c0 RDI: ff0081007f8036c0
Mar 28 13:47:04 (none) kernel: RBP: 0000000000000286 R08: 00000000f8a1426c R09: 0000000000000000
Mar 28 13:47:04 (none) kernel: R10: ffff810043e042c0 R11: ffffffff802fdbb8 R12: ffff8100198d3000
Mar 28 13:47:04 (none) kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 000000000000020c
Mar 28 13:47:04 (none) kernel: FS:  0000000040800950(0063) GS:ffff81007f876cc0(0000) knlGS:00000000b732c9a0
Mar 28 13:47:04 (none) kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 28 13:47:04 (none) kernel: CR2: 00002aaaafc04150 CR3: 0000000070dd7000 CR4: 00000000000006e0
Mar 28 13:47:04 (none) kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 28 13:47:04 (none) kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Mar 28 13:47:04 (none) kernel: Process ktorrent (pid: 5733, threadinfo ffff8100249c6000, task ffff81005f86a840)
Mar 28 13:47:04 (none) kernel: Stack:  ffff810043e042c0 ffff810043e042c0 000000000000020c ffffffff80428a36
Mar 28 13:47:04 (none) kernel:  ffff810024999880 ffffffff804665be 0000000000000000 000000005f86a840
Mar 28 13:47:04 (none) kernel:  ffff810024999d28 0000000000100100 ffff810024999c80 ffff810024999930
Mar 28 13:47:04 (none) kernel: Call Trace:
Mar 28 13:47:04 (none) kernel:  [<ffffffff80428a36>] __kfree_skb+0x9/0x6f
Mar 28 13:47:04 (none) kernel:  [<ffffffff804665be>] tcp_recvmsg+0x614/0x808
Mar 28 13:47:04 (none) kernel:  [<ffffffff80424e56>] sock_common_recvmsg+0x30/0x45
Mar 28 13:47:04 (none) kernel:  [<ffffffff804235f6>] sock_recvmsg+0xf0/0x10f
Mar 28 13:47:04 (none) kernel:  [<ffffffff80231288>] default_wake_function+0x0/0xe
Mar 28 13:47:04 (none) kernel:  [<ffffffff80249e8d>] autoremove_wake_function+0x0/0x2e
Mar 28 13:47:04 (none) kernel:  [<ffffffff80231288>] default_wake_function+0x0/0xe
Mar 28 13:47:04 c-76-22-167-36 syslog-ng[2578]: last message repeated 2 times
Mar 28 13:47:04 (none) kernel:  [<ffffffff80251998>] do_futex+0x8d/0xa3d
Mar 28 13:47:04 (none) kernel:  [<ffffffff804246ce>] sys_recvfrom+0xe2/0x130
Mar 28 13:47:04 (none) kernel:  [<ffffffff80425572>] release_sock+0x13/0x9a
Mar 28 13:47:04 (none) kernel:  [<ffffffff80464c80>] tcp_ioctl+0x11a/0x126
Mar 28 13:47:04 (none) kernel:  [<ffffffff80422ddb>] sock_ioctl+0x1dc/0x200
Mar 28 13:47:04 (none) kernel:  [<ffffffff80295451>] do_ioctl+0x21/0x6b
Mar 28 13:47:04 (none) kernel:  [<ffffffff8020beee>] system_call+0x7e/0x83
Mar 28 13:47:04 (none) kernel:
Mar 28 13:47:04 (none) kernel:
Mar 28 13:47:04 (none) kernel:
Mar 28 13:47:04 (none) kernel: Code: 48 8b 1c c7 8b 13 3b 53 04 73 0c 89 d0 4c 89 64 c3 18 8d 42
Mar 28 13:47:04 (none) kernel: RIP  [<ffffffff802866c1>] kfree+0x6c/0x9f
Mar 28 13:47:04 (none) kernel:  RSP <ffff8100249c7ba8>
Mar 28 13:47:04 (none) kernel: ---[ end trace 5b58994d2438c622 ]---
Mar 28 13:49:37 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Mar 28 13:50:53 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Mar 28 13:52:48 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Mar 28 13:54:21 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Mar 28 13:58:44 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Mar 28 13:59:46 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Mar 28 14:03:16 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Mar 28 14:07:00 (none) kernel: possible SYN flooding on port 6881. Sending cookies.
Comment 1 Anonymous Emailer 2008-03-28 12:27:35 UTC
Reply-To: akpm@linux-foundation.org


(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Fri, 28 Mar 2008 12:17:43 -0700 (PDT)
bugme-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=10350
> 
>            Summary: SYN flooding crashed the kernel?
>            Product: Networking
>            Version: 2.5
>      KernelVersion: 2.6.24.2-default #1 SMP
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: high
>           Priority: P1
>          Component: Netfilter/Iptables
>         AssignedTo: networking_netfilter-iptables@kernel-bugs.osdl.org
>         ReportedBy: harterc1@comcast.net
> 
> 
> Latest working kernel version:
> Earliest failing kernel version:
> Distribution: Suse 10.3 untainted
> Hardware Environment: AMD
> Software Environment: KDE
> Problem Description: kernel crashed when running ktorrent
> 
> Steps to reproduce: haven't yet
> Mar 28 13:40:27 (none) syslog-ng[2578]: STATS: dropped 0
> Mar 28 13:47:04 (none) kernel: general protection fault: 0000 [1] SMP
> Mar 28 13:47:04 (none) kernel: CPU 1
> Mar 28 13:47:04 (none) kernel: Modules linked in: af_packet snd_pcm_oss
> snd_mixer_oss snd_seq snd_seq_device ip6t_REJECT cpufreq_conservative
> cpufreq_ondemand cpufreq_userspace cpufreq_powersave powernow_k8
> ip6table_mangle freq_table ip6table_filter ip6_tables ipv6 fuse dm_crypt loop
> dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd soundcore
> i2c_nforce2 button snd_page_alloc forcedeth i2c_core sg sd_mod edd fan
> generic
> thermal processor
> Mar 28 13:47:04 (none) kernel: Pid: 5733, comm: ktorrent Not tainted
> 2.6.24.2-default #1
> Mar 28 13:47:04 (none) kernel: RIP: 0010:[<ffffffff802866c1>] 
> [<ffffffff802866c1>] kfree+0x6c/0x9f
> Mar 28 13:47:04 (none) kernel: RSP: 0018:ffff8100249c7ba8  EFLAGS: 00010082
> Mar 28 13:47:04 (none) kernel: RAX: 0000000000000001 RBX: ffff810001000000
> RCX:
> 0000000000000001
> Mar 28 13:47:04 (none) kernel: RDX: ffff810001596e28 RSI: ffff810043e042c0
> RDI:
> ff0081007f8036c0
> Mar 28 13:47:04 (none) kernel: RBP: 0000000000000286 R08: 00000000f8a1426c
> R09:
> 0000000000000000
> Mar 28 13:47:04 (none) kernel: R10: ffff810043e042c0 R11: ffffffff802fdbb8
> R12:
> ffff8100198d3000
> Mar 28 13:47:04 (none) kernel: R13: 0000000000000000 R14: 0000000000000000
> R15:
> 000000000000020c
> Mar 28 13:47:04 (none) kernel: FS:  0000000040800950(0063)
> GS:ffff81007f876cc0(0000) knlGS:00000000b732c9a0
> Mar 28 13:47:04 (none) kernel: CS:  0010 DS: 0000 ES: 0000 CR0:
> 0000000080050033
> Mar 28 13:47:04 (none) kernel: CR2: 00002aaaafc04150 CR3: 0000000070dd7000
> CR4:
> 00000000000006e0
> Mar 28 13:47:04 (none) kernel: DR0: 0000000000000000 DR1: 0000000000000000
> DR2:
> 0000000000000000
> Mar 28 13:47:04 (none) kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
> DR7:
> 0000000000000400
> Mar 28 13:47:04 (none) kernel: Process ktorrent (pid: 5733, threadinfo
> ffff8100249c6000, task ffff81005f86a840)
> Mar 28 13:47:04 (none) kernel: Stack:  ffff810043e042c0 ffff810043e042c0
> 000000000000020c ffffffff80428a36
> Mar 28 13:47:04 (none) kernel:  ffff810024999880 ffffffff804665be
> 0000000000000000 000000005f86a840
> Mar 28 13:47:04 (none) kernel:  ffff810024999d28 0000000000100100
> ffff810024999c80 ffff810024999930
> Mar 28 13:47:04 (none) kernel: Call Trace:
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80428a36>] __kfree_skb+0x9/0x6f
> Mar 28 13:47:04 (none) kernel:  [<ffffffff804665be>] tcp_recvmsg+0x614/0x808
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80424e56>]
> sock_common_recvmsg+0x30/0x45
> Mar 28 13:47:04 (none) kernel:  [<ffffffff804235f6>] sock_recvmsg+0xf0/0x10f
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80231288>]
> default_wake_function+0x0/0xe
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80249e8d>]
> autoremove_wake_function+0x0/0x2e
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80231288>]
> default_wake_function+0x0/0xe
> Mar 28 13:47:04 c-76-22-167-36 syslog-ng[2578]: last message repeated 2 times
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80251998>] do_futex+0x8d/0xa3d
> Mar 28 13:47:04 (none) kernel:  [<ffffffff804246ce>] sys_recvfrom+0xe2/0x130
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80425572>] release_sock+0x13/0x9a
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80464c80>] tcp_ioctl+0x11a/0x126
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80422ddb>] sock_ioctl+0x1dc/0x200
> Mar 28 13:47:04 (none) kernel:  [<ffffffff80295451>] do_ioctl+0x21/0x6b
> Mar 28 13:47:04 (none) kernel:  [<ffffffff8020beee>] system_call+0x7e/0x83
> Mar 28 13:47:04 (none) kernel:
> Mar 28 13:47:04 (none) kernel:
> Mar 28 13:47:04 (none) kernel:
> Mar 28 13:47:04 (none) kernel: Code: 48 8b 1c c7 8b 13 3b 53 04 73 0c 89 d0
> 4c
> 89 64 c3 18 8d 42
> Mar 28 13:47:04 (none) kernel: RIP  [<ffffffff802866c1>] kfree+0x6c/0x9f
> Mar 28 13:47:04 (none) kernel:  RSP <ffff8100249c7ba8>
> Mar 28 13:47:04 (none) kernel: ---[ end trace 5b58994d2438c622 ]---
> Mar 28 13:49:37 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> Mar 28 13:50:53 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> Mar 28 13:52:48 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> Mar 28 13:54:21 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> Mar 28 13:58:44 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> Mar 28 13:59:46 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> Mar 28 14:03:16 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> Mar 28 14:07:00 (none) kernel: possible SYN flooding on port 6881. Sending
> cookies.
> 

So all the syn-flooding messages came _after_ the crash?

If so, the messages are possibly a consequence of the crash - the
networking state was left screwed up.

If this happened a single time on a single machine then perhaps you have an
intermittent hardware failure - we'll probably wait this one out, see if
other machines exhibit it, or if a means of reproducing it emerges.
Comment 2 Anonymous Emailer 2008-03-28 12:44:56 UTC
Reply-To: donharter@comcast.net

I have seen some messages like this in my log, but I don't know if there 
is actually a problem or a problem with smartd
Mar 28 11:40:26 (none) syslog-ng[2578]: STATS: dropped 0
Mar 28 11:40:39 (none) smartd[3315]: Device: /dev/sdb, SMART Prefailure 
Attribute: 1 Raw_Read_Error_Rate changed from 96 to 9
4
Mar 28 12:10:38 (none) smartd[3315]: Device: /dev/sdb, SMART Prefailure 
Attribute: 1 Raw_Read_Error_Rate changed from 94 to 9
5
Mar 28 12:10:38 (none) smartd[3315]: Device: /dev/sdb, SMART Usage 
Attribute: 195 Hardware_ECC_Recovered changed from 48 to 4
7
Mar 28 12:40:26 (none) syslog-ng[2578]: STATS: dropped 0
Mar 28 12:40:39 (none) smartd[3315]: Device: /dev/sdb, SMART Prefailure 
Attribute: 1 Raw_Read_Error_Rate changed from 95 to 9
6
Mar 28 12:40:39 (none) smartd[3315]: Device: /dev/sdb, SMART Usage 
Attribute: 195 Hardware_ECC_Recovered changed from 47 to 4
8
Mar 28 13:10:39 (none) smartd[3315]: Device: /dev/sdb, SMART Prefailure 
Attribute: 1 Raw_Read_Error_Rate changed from 96 to 9
8
Mar 28 13:10:39 (none) smartd[3315]: Device: /dev/sdb, SMART Usage 
Attribute: 195 Hardware_ECC_Recovered changed from 48 to 4
7
Mar 28 13:40:27 (none) syslog-ng[2578]: STATS: dropped 0
Mar 28 13:47:04 (none) kernel: general protection fault: 0000 [1] SMP
Mar 28 13:47:04 (none) kernel: CPU 1

You see I get error messages like this from smartd even though that 
parameter is specified in the config file.
Mar 28 14:25:39 (none) smartd[3399]: Device: /dev/sda, opened
Mar 28 14:25:40 (none) smartd[3399]: Device /dev/sda: ATA disk detected 
behind SAT layer
Mar 28 14:25:40 (none) smartd[3399]:   Try adding '-d sat' to the device 
line in the smartd.conf file.
Mar 28 14:25:40 (none) smartd[3399]:   For example: '/dev/sda -a -d sat'
Mar 28 14:25:40 (none) smartd[3399]: Device: /dev/sdb, opened
Mar 28 14:25:40 (none) smartd[3399]: Device /dev/sdb: ATA disk detected 
behind SAT layer
Mar 28 14:25:40 (none) smartd[3399]:   Try adding '-d sat' to the device 
line in the smartd.conf file.
Mar 28 14:25:40 (none) smartd[3399]:   For example: '/dev/sdb -a -d sat'
Mar 28 14:25:40 (none) smartd[3399]: Device: /dev/sda, opened
Mar 28 14:25:40 (none) sshd[3423]: Server listening on :: port 22.

It causes me to suspect that things are not stable with the nforce 4 
drivers.
Anyways I wasn't accessing this /dev/sdb.  I use that drive only for 
backups.
Perhaps I should open up the case and reseat any cards and memory.

After the crash I was still able to submit the bug report.

My iptables config is partly:
root:~>iptables -L
Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere            state 
RELATED,ESTABLISHED
input_ext  all  --  anywhere             anywhere
input_ext  all  --  anywhere             anywhere
LOG        all  --  anywhere             anywhere            limit: avg 
3/min burst 5 LOG level warning tcp-options ip-options prefix 
`SFW2-IN-ILL-TARGET '
DROP       all  --  anywhere             anywhere
This is generated by the Susefirewall2.  I thought I understood iptables 
but it seems that he first statement accepts all traffic and none of the 
others get executed.  I took that statement out once and my web browser 
stopped functioning.  I wasn't running apparmor and perhaps that plays a 
role somewhere.  I may reboot and try to run the kernel as what suse 
calls "failsafe".

Andrew Morton wrote:
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
>
> On Fri, 28 Mar 2008 12:17:43 -0700 (PDT)
> bugme-daemon@bugzilla.kernel.org wrote:
>
>   
>> http://bugzilla.kernel.org/show_bug.cgi?id=10350
>>
>>            Summary: SYN flooding crashed the kernel?
>>            Product: Networking
>>            Version: 2.5
>>      KernelVersion: 2.6.24.2-default #1 SMP
>>           Platform: All
>>         OS/Version: Linux
>>               Tree: Mainline
>>             Status: NEW
>>           Severity: high
>>           Priority: P1
>>          Component: Netfilter/Iptables
>>         AssignedTo: networking_netfilter-iptables@kernel-bugs.osdl.org
>>         ReportedBy: harterc1@comcast.net
>>
>>
>> Latest working kernel version:
>> Earliest failing kernel version:
>> Distribution: Suse 10.3 untainted
>> Hardware Environment: AMD
>> Software Environment: KDE
>> Problem Description: kernel crashed when running ktorrent
>>
>> Steps to reproduce: haven't yet
>> Mar 28 13:40:27 (none) syslog-ng[2578]: STATS: dropped 0
>> Mar 28 13:47:04 (none) kernel: general protection fault: 0000 [1] SMP
>> Mar 28 13:47:04 (none) kernel: CPU 1
>> Mar 28 13:47:04 (none) kernel: Modules linked in: af_packet snd_pcm_oss
>> snd_mixer_oss snd_seq snd_seq_device ip6t_REJECT cpufreq_conservative
>> cpufreq_ondemand cpufreq_userspace cpufreq_powersave powernow_k8
>> ip6table_mangle freq_table ip6table_filter ip6_tables ipv6 fuse dm_crypt
>> loop
>> dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd soundcore
>> i2c_nforce2 button snd_page_alloc forcedeth i2c_core sg sd_mod edd fan
>> generic
>> thermal processor
>> Mar 28 13:47:04 (none) kernel: Pid: 5733, comm: ktorrent Not tainted
>> 2.6.24.2-default #1
>> Mar 28 13:47:04 (none) kernel: RIP: 0010:[<ffffffff802866c1>] 
>> [<ffffffff802866c1>] kfree+0x6c/0x9f
>> Mar 28 13:47:04 (none) kernel: RSP: 0018:ffff8100249c7ba8  EFLAGS: 00010082
>> Mar 28 13:47:04 (none) kernel: RAX: 0000000000000001 RBX: ffff810001000000
>> RCX:
>> 0000000000000001
>> Mar 28 13:47:04 (none) kernel: RDX: ffff810001596e28 RSI: ffff810043e042c0
>> RDI:
>> ff0081007f8036c0
>> Mar 28 13:47:04 (none) kernel: RBP: 0000000000000286 R08: 00000000f8a1426c
>> R09:
>> 0000000000000000
>> Mar 28 13:47:04 (none) kernel: R10: ffff810043e042c0 R11: ffffffff802fdbb8
>> R12:
>> ffff8100198d3000
>> Mar 28 13:47:04 (none) kernel: R13: 0000000000000000 R14: 0000000000000000
>> R15:
>> 000000000000020c
>> Mar 28 13:47:04 (none) kernel: FS:  0000000040800950(0063)
>> GS:ffff81007f876cc0(0000) knlGS:00000000b732c9a0
>> Mar 28 13:47:04 (none) kernel: CS:  0010 DS: 0000 ES: 0000 CR0:
>> 0000000080050033
>> Mar 28 13:47:04 (none) kernel: CR2: 00002aaaafc04150 CR3: 0000000070dd7000
>> CR4:
>> 00000000000006e0
>> Mar 28 13:47:04 (none) kernel: DR0: 0000000000000000 DR1: 0000000000000000
>> DR2:
>> 0000000000000000
>> Mar 28 13:47:04 (none) kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
>> DR7:
>> 0000000000000400
>> Mar 28 13:47:04 (none) kernel: Process ktorrent (pid: 5733, threadinfo
>> ffff8100249c6000, task ffff81005f86a840)
>> Mar 28 13:47:04 (none) kernel: Stack:  ffff810043e042c0 ffff810043e042c0
>> 000000000000020c ffffffff80428a36
>> Mar 28 13:47:04 (none) kernel:  ffff810024999880 ffffffff804665be
>> 0000000000000000 000000005f86a840
>> Mar 28 13:47:04 (none) kernel:  ffff810024999d28 0000000000100100
>> ffff810024999c80 ffff810024999930
>> Mar 28 13:47:04 (none) kernel: Call Trace:
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80428a36>] __kfree_skb+0x9/0x6f
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff804665be>] tcp_recvmsg+0x614/0x808
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80424e56>]
>> sock_common_recvmsg+0x30/0x45
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff804235f6>] sock_recvmsg+0xf0/0x10f
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80231288>]
>> default_wake_function+0x0/0xe
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80249e8d>]
>> autoremove_wake_function+0x0/0x2e
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80231288>]
>> default_wake_function+0x0/0xe
>> Mar 28 13:47:04 c-76-22-167-36 syslog-ng[2578]: last message repeated 2
>> times
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80251998>] do_futex+0x8d/0xa3d
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff804246ce>] sys_recvfrom+0xe2/0x130
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80425572>] release_sock+0x13/0x9a
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80464c80>] tcp_ioctl+0x11a/0x126
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80422ddb>] sock_ioctl+0x1dc/0x200
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff80295451>] do_ioctl+0x21/0x6b
>> Mar 28 13:47:04 (none) kernel:  [<ffffffff8020beee>] system_call+0x7e/0x83
>> Mar 28 13:47:04 (none) kernel:
>> Mar 28 13:47:04 (none) kernel:
>> Mar 28 13:47:04 (none) kernel:
>> Mar 28 13:47:04 (none) kernel: Code: 48 8b 1c c7 8b 13 3b 53 04 73 0c 89 d0
>> 4c
>> 89 64 c3 18 8d 42
>> Mar 28 13:47:04 (none) kernel: RIP  [<ffffffff802866c1>] kfree+0x6c/0x9f
>> Mar 28 13:47:04 (none) kernel:  RSP <ffff8100249c7ba8>
>> Mar 28 13:47:04 (none) kernel: ---[ end trace 5b58994d2438c622 ]---
>> Mar 28 13:49:37 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>> Mar 28 13:50:53 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>> Mar 28 13:52:48 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>> Mar 28 13:54:21 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>> Mar 28 13:58:44 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>> Mar 28 13:59:46 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>> Mar 28 14:03:16 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>> Mar 28 14:07:00 (none) kernel: possible SYN flooding on port 6881. Sending
>> cookies.
>>
>>     
>
> So all the syn-flooding messages came _after_ the crash?
>
> If so, the messages are possibly a consequence of the crash - the
> networking state was left screwed up.
>
> If this happened a single time on a single machine then perhaps you have an
> intermittent hardware failure - we'll probably wait this one out, see if
> other machines exhibit it, or if a means of reproducing it emerges.
>
>
>   
Comment 3 Don Harter 2008-03-30 09:05:06 UTC
I saw no syn flooding messages in the log so this event may be one of several that trigger this.  "iptables -vL" tells what really goes on.  At one time port 6881 was not protected from flooding.  Here is a crash report:
general protection fault: 0000 [1] SMP 

CPU 1 

Modules linked in: af_packet snd_pcm_oss snd_mixer_oss snd_seq snd_seq_device ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables ipv6 fuse dm_crypt loop dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd soundcore snd_page_alloc i2c_nforce2 forcedeth i2c_core sg sd_mod edd generic

Pid: 0, comm: swapper Not tainted 2.6.24.2-default #1

RIP: 0010:[<ffffffff802866c1>]  [<ffffffff802866c1>] kfree+0x6c/0x9f

RSP: 0018:ffff81007f87bc70  EFLAGS: 00010082

RAX: 0000000000000001 RBX: ffff810001000000 RCX: 0000000000000001

RDX: ffff810002b9eea8 RSI: 0000000000000004 RDI: ff0081007f8036c0

RBP: 0000000000000286 R08: 00000000000000b3 R09: ffff81005e9c47d8

R10: ffff8100508f2d00 R11: 0000000000000286 R12: ffff81007e443000

R13: 00000000a8fa1e81 R14: 0000000000000014 R15: ffff81007e443032

FS:  00002aae4a74cd70(0000) GS:ffff81007f851f40(0000) knlGS:00000000b5208b90

CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b

CR2: 00002aaab2041000 CR3: 000000005e9b1000 CR4: 00000000000006e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

Process swapper (pid: 0, threadinfo ffff81007f876000, task ffff81007f8527c0)

Stack:  ffff8100698bf6c0 ffff8100508f2d00 ffff8100698bf6c0 ffffffff80428a36

 ffff8100508f2d00 ffffffff8046d568 ffff81005e9c47d8 ffff8100698bf6f8

 00148100698bf6c0 ffff8100508f2d00 ffff8100698bf6c0 ffff8100698bf6c0

Call Trace:

 <IRQ>  [<ffffffff80428a36>] __kfree_skb+0x9/0x6f

 [<ffffffff8046d568>] tcp_rcv_established+0xef/0x717

 [<ffffffff80473327>] tcp_v4_do_rcv+0x30/0x39c

 [<ffffffff8048c2bb>] ipv4_confirm+0x3f/0x45

 [<ffffffff804759c6>] tcp_v4_rcv+0x8e4/0x94e

 [<ffffffff8045aee7>] ip_local_deliver_finish+0x0/0x212

 [<ffffffff8045b039>] ip_local_deliver_finish+0x152/0x212

 [<ffffffff8045aec4>] ip_rcv_finish+0x2f8/0x31b

 [<ffffffff8045b416>] ip_rcv+0x239/0x27e

 [<ffffffff8042dd4a>] netif_receive_skb+0x3ae/0x3cf

 [<ffffffff80430366>] process_backlog+0x7c/0xda

 [<ffffffff8042fdc0>] net_rx_action+0xb7/0x1ac

 [<ffffffff8023c9cd>] __do_softirq+0x65/0xcf

 [<ffffffff8021e4aa>] ack_apic_level+0x10/0xd9

 [<ffffffff8020d07c>] call_softirq+0x1c/0x28

 [<ffffffff8020e4d0>] do_softirq+0x2c/0x7d

 [<ffffffff8020e791>] do_IRQ+0x13e/0x161

 [<ffffffff8020b0bf>] default_idle+0x0/0x3d

 [<ffffffff8020b0bf>] default_idle+0x0/0x3d

 [<ffffffff8020c401>] ret_from_intr+0x0/0xa

 <EOI>  [<ffffffff804a81b3>] thread_return+0xa7/0xab

 [<ffffffff8020b0e8>] default_idle+0x29/0x3d

 [<ffffffff8020b18a>] cpu_idle+0x8e/0xb1





Code: 48 8b 1c c7 8b 13 3b 53 04 73 0c 89 d0 4c 89 64 c3 18 8d 42 

RIP  [<ffffffff802866c1>] kfree+0x6c/0x9f

 RSP <ffff81007f87bc70>

---[ end trace 25880fe3df922ae1 ]---

Kernel panic - not syncing: Aiee, killing interrupt handler!
Comment 4 Don Harter 2008-03-30 09:06:56 UTC
Here is another crash:
general protection fault: 0000 [1] SMP 

CPU 1 

Modules linked in: nls_iso8859_1 nls_cp437 vfat fat af_packet snd_pcm_oss snd_mixer_oss snd_seq snd_seq_device ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables ipv6 fuse dm_crypt loop dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd soundcore snd_page_alloc forcedeth i2c_nforce2 i2c_core sg sd_mod edd generic

Pid: 5032, comm: beagled-helper Not tainted 2.6.24.2-default #1

RIP: 0010:[<ffffffff80270338>]  [<ffffffff80270338>] put_page+0x65/0xb3

RSP: 0000:ffff81002a98dd78  EFLAGS: 00010047

RAX: ffff810001e9be58 RBX: ffff810002150fb8 RCX: ffff810002150fe0

RDX: ff00810002150fa8 RSI: 0000000000000001 RDI: ffffffff805bf6c0

RBP: ffffffff805be280 R08: ffffffff805be384 R09: 0000000000000bc2

R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000212

R13: ffff810002150fb8 R14: ffff81007f912f40 R15: ffff81007c51e3d8

FS:  0000000040fba950(0063) GS:ffff81007f851f40(0000) knlGS:00000000b73c39a0

CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00002aaaaf91d000 CR3: 0000000057cac000 CR4: 00000000000006e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

Process beagled-helper (pid: 5032, threadinfo ffff81002a98c000, task ffff81002c90f080)

Stack:  0000000000000008 ffff810002150fb8 ffff81005e1918e8 ffffffff80275255

 ffff8100024957c8 ffff810057c60be0 00002aaaaf91d000 ffff81005e1918e8

 ffff8100024957c8 800000004f291065 ffff81002c90f080 ffff81007c51e3d8

Call Trace:

 [<ffffffff80275255>] do_wp_page+0x447/0x4bc

 [<ffffffff802769dc>] handle_mm_fault+0x69a/0x6f8

 [<ffffffff80291e6e>] may_open+0xaf/0x20b

 [<ffffffff802943ce>] open_namei+0x2ca/0x649

 [<ffffffff804ab45d>] do_page_fault+0x338/0x6c1

 [<ffffffff8028ce6d>] sys_newfstat+0x20/0x29

 [<ffffffff804a9ae9>] error_exit+0x0/0x51





Code: 48 89 42 08 48 89 10 48 c7 41 08 00 02 20 00 8b 03 48 c7 43 

RIP  [<ffffffff80270338>] put_page+0x65/0xb3

 RSP <ffff81002a98dd78>

---[ end trace 95eccb101e72a483 ]---
Comment 5 Don Harter 2008-03-30 13:50:32 UTC
Here is another crash:
general protection fault: 0000 [1] SMP 

CPU 0 

Modules linked in: nls_iso8859_1 nls_cp437 vfat fat af_packet snd_pcm_oss snd_mixer_oss snd_seq snd_seq_device ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables ipv6 fuse dm_crypt loop dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer snd i2c_nforce2 soundcore forcedeth snd_page_alloc i2c_core sg sd_mod edd generic

Pid: 28723, comm: beagled-helper Not tainted 2.6.24.2-default #1

RIP: 0010:[<ffffffff80270338>]  [<ffffffff80270338>] put_page+0x65/0xb3

RSP: 0000:ffff810065627d78  EFLAGS: 00010047

RAX: ffff8100023e00d8 RBX: ffff810002395fb8 RCX: ffff810002395fe0

RDX: ff00810002485860 RSI: 0000000000000001 RDI: ffffffff805bf6c0

RBP: ffffffff805be280 R08: ffffffff805be304 R09: 0000000000000982

R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000212

R13: ffff810002395fb8 R14: ffff81007aca2e80 R15: ffff810016d083d8

FS:  0000000040f8a950(0063) GS:ffffffff805f2000(0000) knlGS:00000000b5b52b90

CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b

CR2: 00002aaaaf8b1000 CR3: 0000000077c12000 CR4: 00000000000006e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

Process beagled-helper (pid: 28723, threadinfo ffff810065626000, task ffff810078019840)

Stack:  0000000000000008 ffff810002395fb8 ffff810077d74588 ffffffff80275255

 ffff810002a37170 ffff810077d9abe0 00002aaaaf8b1000 ffff810077d74588

 ffff810002a37170 8000000059891065 ffff810078019840 ffff810016d083d8

Call Trace:

 [<ffffffff80275255>] do_wp_page+0x447/0x4bc

 [<ffffffff802769dc>] handle_mm_fault+0x69a/0x6f8

 [<ffffffff804ab45d>] do_page_fault+0x338/0x6c1

 [<ffffffff804a8146>] thread_return+0x3a/0xab

 [<ffffffff8028a47f>] vfs_write+0x13a/0x14f

 [<ffffffff804a9ae9>] error_exit+0x0/0x51
Comment 6 Don Harter 2008-04-06 10:06:12 UTC
Here is the same bug in a more recent kernel.
root:~>uname -a
Linux c-76-22-168-235 2.6.24.4-default #1 SMP Sun Mar 30 17:12:02 CDT 2008 x86_64 x86_64 x86_64 GNU/Linux

Apr  5 21:47:08 c-76-22-168-235 kernel: general protection fault: 0000 [1] SMP
Apr  5 21:47:08 c-76-22-168-235 syslog-ng[2587]: Changing permissions on special file /dev/xconsole
Apr  5 21:47:08 c-76-22-168-235 syslog-ng[2587]: Changing permissions on special file /dev/tty10
Apr  5 21:47:08 c-76-22-168-235 kernel: CPU 1
Apr  5 21:47:08 c-76-22-168-235 kernel: Modules linked in: nls_utf8 af_packet snd_pcm_oss snd_mixer_oss snd_seq snd_seq_devic
e ip6t_REJECT cpufreq_conservative cpufreq_ondemand ip6table_mangle cpufreq_userspace cpufreq_powersave powernow_k8 freq_tabl
e ip6table_filter ip6_tables ipv6 fuse dm_crypt loop dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer i2c_nforce
2 snd forcedeth i2c_core soundcore button snd_page_alloc sg sd_mod edd fan generic thermal processor
Apr  5 21:47:08 c-76-22-168-235 kernel: Pid: 26249, comm: ktorrent Not tainted 2.6.24.4-default #1
Apr  5 21:47:08 c-76-22-168-235 kernel: RIP: 0010:[<ffffffff80286979>]  [<ffffffff80286979>] kfree+0x6c/0x9f
Apr  5 21:47:08 c-76-22-168-235 kernel: RSP: 0018:ffff81001d773ba8  EFLAGS: 00010082
Apr  5 21:47:08 c-76-22-168-235 kernel: RAX: 0000000000000001 RBX: ffff810001000000 RCX: 0000000000000001
Apr  5 21:47:08 c-76-22-168-235 kernel: RDX: ffff810002b9e7a8 RSI: ffff81006649be80 RDI: ff0081007f8036c0
Apr  5 21:47:08 c-76-22-168-235 kernel: RBP: 0000000000000286 R08: 00000000c9769298 R09: 0000000000000000
Apr  5 21:47:08 c-76-22-168-235 kernel: R10: ffff81006649be80 R11: ffffffff802fde90 R12: ffff81007e423000
Apr  5 21:47:08 c-76-22-168-235 kernel: R13: 0000000000000000 R14: 00000000000005b4 R15: 0000000000002238
Apr  5 21:47:08 c-76-22-168-235 kernel: FS:  0000000040800950(0063) GS:ffff81007f876cc0(0000) knlGS:00000000b5a9fb90
Apr  5 21:47:08 c-76-22-168-235 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr  5 21:47:08 c-76-22-168-235 kernel: CR2: 00002aaaaae190c4 CR3: 000000000a52f000 CR4: 00000000000006e0
Apr  5 21:47:08 c-76-22-168-235 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr  5 21:47:08 c-76-22-168-235 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Apr  5 21:47:08 c-76-22-168-235 kernel: Process ktorrent (pid: 26249, threadinfo ffff81001d772000, task ffff8100250f57c0)
Apr  5 21:47:08 c-76-22-168-235 kernel: Stack:  ffff81006649be80 ffff81006649be80 00000000000005b4 ffffffff80428cca
Apr  5 21:47:08 c-76-22-168-235 kernel:  ffff810025179340 ffffffff8046683e ffff810000000000 00000000250f57c0
Apr  5 21:47:08 c-76-22-168-235 kernel:  ffff8100251797e8 0000000000100100 ffff810025179740 ffff8100251793f0
Apr  5 21:47:08 c-76-22-168-235 kernel: Call Trace:
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80428cca>] __kfree_skb+0x9/0x6f
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff8046683e>] tcp_recvmsg+0x614/0x808
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff804250ea>] sock_common_recvmsg+0x30/0x45
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff8042388a>] sock_recvmsg+0xf0/0x10f
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80231290>] default_wake_function+0x0/0xe
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80249e4d>] autoremove_wake_function+0x0/0x2e
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80231290>] default_wake_function+0x0/0xe
Apr  5 21:47:08 c-76-22-168-235 syslog-ng[2587]: last message repeated 2 times
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80251972>] do_futex+0x87/0xa55
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80424962>] sys_recvfrom+0xe2/0x130
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80425806>] release_sock+0x13/0x9a
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff80464f00>] tcp_ioctl+0x11a/0x126
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff8042306f>] sock_ioctl+0x1dc/0x200
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff8029570d>] do_ioctl+0x21/0x6b
Apr  5 21:47:08 c-76-22-168-235 kernel:  [<ffffffff8020beee>] system_call+0x7e/0x83
Apr  5 21:47:08 c-76-22-168-235 kernel:
Apr  5 21:47:08 c-76-22-168-235 kernel:
Apr  5 21:47:08 c-76-22-168-235 kernel: Code: 48 8b 1c c7 8b 13 3b 53 04 73 0c 89 d0 4c 89 64 c3 18 8d 42
Apr  5 21:47:08 c-76-22-168-235 kernel: RIP  [<ffffffff80286979>] kfree+0x6c/0x9f
Apr  5 21:47:08 c-76-22-168-235 kernel:  RSP <ffff81001d773ba8>
Apr  5 21:47:08 c-76-22-168-235 kernel: ---[ end trace 671e4e85663660df ]---

Apr  5 21:50:02 c-76-22-168-235 kernel: possible SYN flooding on port 6881. Sending cookies.
Apr  5 21:51:23 c-76-22-168-235 kernel: possible SYN flooding on port 6881. Sending cookies.
Comment 7 Alan 2012-05-18 10:43:40 UTC
Please re-open if seen with a modern kernel

Note You need to log in before you can comment on or make changes to this bug.