Bug 10996

Summary: 2.6.25 kernels spew SKB BUG: Invalid truesize when using ipsec/openswan
Product: Networking Reporter: Tony Lill (ajlill)
Component: IPV4Assignee: Stephen Hemminger (stephen)
Status: CLOSED CODE_FIX    
Severity: low CC: aaron, alan, berni, daniel.rode, hostmaster, jbohac, mdrozdziel, mishu, ole, zdzichu
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 2.6.25.9 Subsystem:
Regression: No Bisected commit-id:

Description Tony Lill 2008-06-27 16:37:21 UTC
Latest working kernel version: 2.6.24.4
Earliest failing kernel version:2.6.25.4
Distribution: Debian etch
Hardware Environment:
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Celeron(R) CPU 2.93GHz
stepping        : 1
cpu MHz         : 2929.700
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe constant_tsc up pebs bts pni monitor ds_cpl cid xtpr
bogomips        : 5865.13
clflush size    : 64

Linux-i686-2.6.25.9 $ cat /proc/modules 
xfrm_user 28040 2 - Live 0xe037d000
xfrm4_tunnel 3976 0 - Live 0xe0226000
af_key 39452 0 - Live 0xe0308000
authenc 8072 5 - Live 0xe0443000
xfrm4_mode_tunnel 3976 10 - Live 0xe0435000
cls_u32 9100 1 - Live 0xe045e000
sch_red 7432 1 - Live 0xe0440000
xt_mark 3208 2 - Live 0xe0433000
xt_layer7 12564 5 - Live 0xe0463000
xt_length 2952 2 - Live 0xe0431000
cls_fw 6536 14 - Live 0xe042b000
sch_sfq 8072 14 - Live 0xe0422000
sch_htb 17288 2 - Live 0xe0425000
xt_TCPMSS 6536 1 - Live 0xe0400000
xt_tcpmss 3208 1 - Live 0xe0418000
netconsole 11044 0 - Live 0xe043c000
configfs 28592 2 netconsole, Live 0xe0446000
pppoe 14564 2 - Live 0xe0437000
pppox 5396 1 pppoe, Live 0xe042e000
iptable_mangle 4360 1 - Live 0xe0415000
xt_MARK 3592 47 - Live 0xe0395000
xt_conntrack 5128 1 - Live 0xe0412000
ipt_REDIRECT 3208 1 - Live 0xe035f000
xt_limit 3596 4 - Live 0xe0334000
ipt_recent 10428 3 - Live 0xe041a000
ipt_REJECT 5000 11 - Live 0xe040f000
xt_state 3464 10 - Live 0xe0306000
xt_tcpudp 4360 203 - Live 0xe040c000
ipt_LOG 7052 27 - Live 0xe03f7000
iptable_filter 4492 1 - Live 0xe03fd000
iptable_nat 8340 1 - Live 0xe0408000
ip_tables 15256 3 iptable_mangle,iptable_filter,iptable_nat, Live 0xe0403000
nf_nat_pptp 4872 0 - Live 0xe03fa000
nf_conntrack_pptp 8076 1 nf_nat_pptp, Live 0xe03a2000
nf_conntrack_proto_gre 7176 1 nf_conntrack_pptp, Live 0xe034b000
nf_nat_proto_gre 3852 1 nf_nat_pptp, Live 0xe0254000
nf_nat_sip 5512 0 - Live 0xe0392000
nf_conntrack_sip 10148 1 nf_nat_sip, Live 0xe039e000
nf_nat_ftp 4364 0 - Live 0xe038f000
nf_nat 20532 6 ipt_REDIRECT,iptable_nat,nf_nat_pptp,nf_nat_proto_gre,nf_nat_sip,
nf_nat_ftp, Live 0xe0397000
nf_conntrack_ipv4 18716 19 iptable_nat,nf_nat, Live 0xe0377000
nf_conntrack_ftp 10292 1 nf_nat_ftp, Live 0xe038b000
nf_conntrack 64396 13 xt_layer7,xt_conntrack,xt_state,iptable_nat,nf_nat_pptp,nf
_conntrack_pptp,nf_conntrack_proto_gre,nf_nat_sip,nf_conntrack_sip,nf_nat_ftp,nf
_nat,nf_conntrack_ipv4,nf_conntrack_ftp, Live 0xe03e6000
xt_TARPIT 5000 2 - Live 0xe0374000
x_tables 17804 17 xt_mark,xt_layer7,xt_length,xt_TCPMSS,xt_tcpmss,xt_MARK,xt_con
ntrack,ipt_REDIRECT,xt_limit,ipt_recent,ipt_REJECT,xt_state,xt_tcpudp,ipt_LOG,ip
table_nat,ip_tables,xt_TARPIT, Live 0xe0385000
tunnel4 4752 1 xfrm4_tunnel, Live 0xe034e000
ipcomp 9744 0 - Live 0xe0370000
esp4 9096 5 - Live 0xe035b000
aead 9096 2 authenc,esp4, Live 0xe0357000
ah4 7432 0 - Live 0xe0341000
af_packet 23176 8 - Live 0xe0369000
ppp_generic 28196 6 pppoe,pppox, Live 0xe0361000
slhc 7176 1 ppp_generic, Live 0xe004c000
ipv6 261772 34 - Live 0xe03a5000
deflate 4872 0 - Live 0xe0327000
zlib_deflate 19776 1 deflate, Live 0xe0351000
ctr 6024 0 - Live 0xe0316000
twofish 8328 0 - Live 0xe0330000
twofish_common 13832 1 twofish, Live 0xe033c000
camellia 22536 0 - Live 0xe0344000
serpent 18696 0 - Live 0xe0336000
blowfish 10376 0 - Live 0xe0323000
des_generic 17288 0 - Live 0xe032a000
cbc 5384 5 - Live 0xe0313000
aes_i586 9352 5 - Live 0xe0241000
aes_generic 31052 1 aes_i586, Live 0xe031a000
xcbc 6416 0 - Live 0xe023e000
sha256_generic 12808 0 - Live 0xe0301000
sha1_generic 3720 5 - Live 0xe017e000
crypto_null 4232 0 - Live 0xe023b000
crypto_blkcipher 19864 9 authenc,ctr,cbc,crypto_null, Live 0xe02fb000
dm_snapshot 20400 0 - Live 0xe024e000
dm_mirror 29200 0 - Live 0xe0245000
dm_mod 63032 2 dm_snapshot,dm_mirror, Live 0xe026e000
w83627hf 25140 0 - Live 0xe0164000
hwmon_vid 3848 1 w83627hf, Live 0xe0052000
hwmon 4388 1 w83627hf, Live 0xe0159000
8250_pnp 10888 0 - Live 0xe0237000
snd_intel8x0 34252 0 - Live 0xe021c000
snd_ac97_codec 93884 1 snd_intel8x0, Live 0xe0256000
ac97_bus 2952 1 snd_ac97_codec, Live 0xe00a6000
8250 26776 1 8250_pnp, Live 0xe0214000
serial_core 22536 1 8250, Live 0xe0177000
floppy 57224 0 - Live 0xe0228000
psmouse 39368 0 - Live 0xe016c000
snd_pcm 73640 2 snd_intel8x0,snd_ac97_codec, Live 0xe0201000
pcspkr 4232 0 - Live 0xe015c000
rtc 15532 0 - Live 0xe013a000
serio_raw 7948 0 - Live 0xe00f6000
i2c_i801 10648 0 - Live 0xe013f000
snd_timer 24600 1 snd_pcm, Live 0xe0143000
i2c_core 24348 1 i2c_i801, Live 0xe010b000
snd 52744 4 snd_intel8x0,snd_ac97_codec,snd_pcm,snd_timer, Live 0xe014b000
soundcore 9188 1 snd, Live 0xe0136000
snd_page_alloc 11412 2 snd_intel8x0,snd_pcm, Live 0xe0132000
rng_core 6156 0 - Live 0xe001e000
iTCO_wdt 12724 0 - Live 0xe00ed000
button 9496 0 - Live 0xe00f2000
intel_agp 27112 1 - Live 0xe0103000
agpgart 35416 1 intel_agp, Live 0xe00f9000
sg 36168 0 - Live 0xe00d4000
evdev 13348 0 - Live 0xe00a1000
sr_mod 17492 0 - Live 0xe00ce000
cdrom 36292 1 sr_mod, Live 0xe008a000
ext3 124888 3 - Live 0xe0112000
jbd 50736 1 ext3, Live 0xe00df000
mbcache 9740 1 ext3, Live 0xe0024000
sd_mod 27296 10 - Live 0xe0094000
pata_acpi 8200 0 - Live 0xe007e000
ata_generic 9104 0 - Live 0xe007a000
ata_piix 22796 10 - Live 0xe0083000
libata 150576 3 pata_acpi,ata_generic,ata_piix, Live 0xe00a8000
dock 12188 1 libata, Live 0xe0061000
8139cp 23440 0 - Live 0xe0073000
8139too 27544 0 - Live 0xe006b000
mii 6408 2 8139cp,8139too, Live 0xe004f000
ehci_hcd 34336 0 - Live 0xe0057000
uhci_hcd 25244 0 - Live 0xe0031000
thermal 21308 0 - Live 0xe0039000
processor 44792 1 thermal, Live 0xe0040000
fan 5256 0 - Live 0xe0021000
unix 29244 136 - Live 0xe0028000

Software Environment:
Debian etch firewall using pppoe and openswan
Problem Description:

The kernel spews hundreds of 
SKB BUG: Invalid truesize (560) len=392, sizeof(sk_buff)=176
With various values for truesize and len whenever ipsec tunnels are up.

No other detectible problems, except log pollution.

Steps to reproduce:
/etc/init.d/ipsec start
Comment 1 Anonymous Emailer 2008-06-27 16:41:10 UTC
Reply-To: akpm@linux-foundation.org


(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Fri, 27 Jun 2008 16:37:22 -0700 (PDT)
bugme-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=10996
> 
>            Summary: 2.6.25 kernels spew SKB BUG: Invalid truesize when using
>                     ipsec/openswan
>            Product: Networking
>            Version: 2.5
>      KernelVersion: Linux version 2.6.25.9 (ajlill@ds) (gcc version 3.4.6
>                     (Debian 3.
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: low
>           Priority: P1
>          Component: IPV4
>         AssignedTo: shemminger@linux-foundation.org
>         ReportedBy: ajlill@ajlc.waterloo.on.ca
> 
> 
> Latest working kernel version: 2.6.24.4
> Earliest failing kernel version:2.6.25.4

These two lines are contradictory.  Please clarify.

> Distribution: Debian etch
> Hardware Environment:
> processor       : 0
> vendor_id       : GenuineIntel
> cpu family      : 15
> model           : 4
> model name      : Intel(R) Celeron(R) CPU 2.93GHz
> stepping        : 1
> cpu MHz         : 2929.700
> cache size      : 256 KB
> fdiv_bug        : no
> hlt_bug         : no
> f00f_bug        : no
> coma_bug        : no
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 5
> wp              : yes
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov
> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe constant_tsc up
> pebs
> bts pni monitor ds_cpl cid xtpr
> bogomips        : 5865.13
> clflush size    : 64
> 
> Linux-i686-2.6.25.9 $ cat /proc/modules 
> xfrm_user 28040 2 - Live 0xe037d000
> xfrm4_tunnel 3976 0 - Live 0xe0226000
> af_key 39452 0 - Live 0xe0308000
> authenc 8072 5 - Live 0xe0443000
> xfrm4_mode_tunnel 3976 10 - Live 0xe0435000
> cls_u32 9100 1 - Live 0xe045e000
> sch_red 7432 1 - Live 0xe0440000
> xt_mark 3208 2 - Live 0xe0433000
> xt_layer7 12564 5 - Live 0xe0463000
> xt_length 2952 2 - Live 0xe0431000
> cls_fw 6536 14 - Live 0xe042b000
> sch_sfq 8072 14 - Live 0xe0422000
> sch_htb 17288 2 - Live 0xe0425000
> xt_TCPMSS 6536 1 - Live 0xe0400000
> xt_tcpmss 3208 1 - Live 0xe0418000
> netconsole 11044 0 - Live 0xe043c000
> configfs 28592 2 netconsole, Live 0xe0446000
> pppoe 14564 2 - Live 0xe0437000
> pppox 5396 1 pppoe, Live 0xe042e000
> iptable_mangle 4360 1 - Live 0xe0415000
> xt_MARK 3592 47 - Live 0xe0395000
> xt_conntrack 5128 1 - Live 0xe0412000
> ipt_REDIRECT 3208 1 - Live 0xe035f000
> xt_limit 3596 4 - Live 0xe0334000
> ipt_recent 10428 3 - Live 0xe041a000
> ipt_REJECT 5000 11 - Live 0xe040f000
> xt_state 3464 10 - Live 0xe0306000
> xt_tcpudp 4360 203 - Live 0xe040c000
> ipt_LOG 7052 27 - Live 0xe03f7000
> iptable_filter 4492 1 - Live 0xe03fd000
> iptable_nat 8340 1 - Live 0xe0408000
> ip_tables 15256 3 iptable_mangle,iptable_filter,iptable_nat, Live 0xe0403000
> nf_nat_pptp 4872 0 - Live 0xe03fa000
> nf_conntrack_pptp 8076 1 nf_nat_pptp, Live 0xe03a2000
> nf_conntrack_proto_gre 7176 1 nf_conntrack_pptp, Live 0xe034b000
> nf_nat_proto_gre 3852 1 nf_nat_pptp, Live 0xe0254000
> nf_nat_sip 5512 0 - Live 0xe0392000
> nf_conntrack_sip 10148 1 nf_nat_sip, Live 0xe039e000
> nf_nat_ftp 4364 0 - Live 0xe038f000
> nf_nat 20532 6
> ipt_REDIRECT,iptable_nat,nf_nat_pptp,nf_nat_proto_gre,nf_nat_sip,
> nf_nat_ftp, Live 0xe0397000
> nf_conntrack_ipv4 18716 19 iptable_nat,nf_nat, Live 0xe0377000
> nf_conntrack_ftp 10292 1 nf_nat_ftp, Live 0xe038b000
> nf_conntrack 64396 13
> xt_layer7,xt_conntrack,xt_state,iptable_nat,nf_nat_pptp,nf
>
> _conntrack_pptp,nf_conntrack_proto_gre,nf_nat_sip,nf_conntrack_sip,nf_nat_ftp,nf
> _nat,nf_conntrack_ipv4,nf_conntrack_ftp, Live 0xe03e6000
> xt_TARPIT 5000 2 - Live 0xe0374000
> x_tables 17804 17
> xt_mark,xt_layer7,xt_length,xt_TCPMSS,xt_tcpmss,xt_MARK,xt_con
>
> ntrack,ipt_REDIRECT,xt_limit,ipt_recent,ipt_REJECT,xt_state,xt_tcpudp,ipt_LOG,ip
> table_nat,ip_tables,xt_TARPIT, Live 0xe0385000
> tunnel4 4752 1 xfrm4_tunnel, Live 0xe034e000
> ipcomp 9744 0 - Live 0xe0370000
> esp4 9096 5 - Live 0xe035b000
> aead 9096 2 authenc,esp4, Live 0xe0357000
> ah4 7432 0 - Live 0xe0341000
> af_packet 23176 8 - Live 0xe0369000
> ppp_generic 28196 6 pppoe,pppox, Live 0xe0361000
> slhc 7176 1 ppp_generic, Live 0xe004c000
> ipv6 261772 34 - Live 0xe03a5000
> deflate 4872 0 - Live 0xe0327000
> zlib_deflate 19776 1 deflate, Live 0xe0351000
> ctr 6024 0 - Live 0xe0316000
> twofish 8328 0 - Live 0xe0330000
> twofish_common 13832 1 twofish, Live 0xe033c000
> camellia 22536 0 - Live 0xe0344000
> serpent 18696 0 - Live 0xe0336000
> blowfish 10376 0 - Live 0xe0323000
> des_generic 17288 0 - Live 0xe032a000
> cbc 5384 5 - Live 0xe0313000
> aes_i586 9352 5 - Live 0xe0241000
> aes_generic 31052 1 aes_i586, Live 0xe031a000
> xcbc 6416 0 - Live 0xe023e000
> sha256_generic 12808 0 - Live 0xe0301000
> sha1_generic 3720 5 - Live 0xe017e000
> crypto_null 4232 0 - Live 0xe023b000
> crypto_blkcipher 19864 9 authenc,ctr,cbc,crypto_null, Live 0xe02fb000
> dm_snapshot 20400 0 - Live 0xe024e000
> dm_mirror 29200 0 - Live 0xe0245000
> dm_mod 63032 2 dm_snapshot,dm_mirror, Live 0xe026e000
> w83627hf 25140 0 - Live 0xe0164000
> hwmon_vid 3848 1 w83627hf, Live 0xe0052000
> hwmon 4388 1 w83627hf, Live 0xe0159000
> 8250_pnp 10888 0 - Live 0xe0237000
> snd_intel8x0 34252 0 - Live 0xe021c000
> snd_ac97_codec 93884 1 snd_intel8x0, Live 0xe0256000
> ac97_bus 2952 1 snd_ac97_codec, Live 0xe00a6000
> 8250 26776 1 8250_pnp, Live 0xe0214000
> serial_core 22536 1 8250, Live 0xe0177000
> floppy 57224 0 - Live 0xe0228000
> psmouse 39368 0 - Live 0xe016c000
> snd_pcm 73640 2 snd_intel8x0,snd_ac97_codec, Live 0xe0201000
> pcspkr 4232 0 - Live 0xe015c000
> rtc 15532 0 - Live 0xe013a000
> serio_raw 7948 0 - Live 0xe00f6000
> i2c_i801 10648 0 - Live 0xe013f000
> snd_timer 24600 1 snd_pcm, Live 0xe0143000
> i2c_core 24348 1 i2c_i801, Live 0xe010b000
> snd 52744 4 snd_intel8x0,snd_ac97_codec,snd_pcm,snd_timer, Live 0xe014b000
> soundcore 9188 1 snd, Live 0xe0136000
> snd_page_alloc 11412 2 snd_intel8x0,snd_pcm, Live 0xe0132000
> rng_core 6156 0 - Live 0xe001e000
> iTCO_wdt 12724 0 - Live 0xe00ed000
> button 9496 0 - Live 0xe00f2000
> intel_agp 27112 1 - Live 0xe0103000
> agpgart 35416 1 intel_agp, Live 0xe00f9000
> sg 36168 0 - Live 0xe00d4000
> evdev 13348 0 - Live 0xe00a1000
> sr_mod 17492 0 - Live 0xe00ce000
> cdrom 36292 1 sr_mod, Live 0xe008a000
> ext3 124888 3 - Live 0xe0112000
> jbd 50736 1 ext3, Live 0xe00df000
> mbcache 9740 1 ext3, Live 0xe0024000
> sd_mod 27296 10 - Live 0xe0094000
> pata_acpi 8200 0 - Live 0xe007e000
> ata_generic 9104 0 - Live 0xe007a000
> ata_piix 22796 10 - Live 0xe0083000
> libata 150576 3 pata_acpi,ata_generic,ata_piix, Live 0xe00a8000
> dock 12188 1 libata, Live 0xe0061000
> 8139cp 23440 0 - Live 0xe0073000
> 8139too 27544 0 - Live 0xe006b000
> mii 6408 2 8139cp,8139too, Live 0xe004f000
> ehci_hcd 34336 0 - Live 0xe0057000
> uhci_hcd 25244 0 - Live 0xe0031000
> thermal 21308 0 - Live 0xe0039000
> processor 44792 1 thermal, Live 0xe0040000
> fan 5256 0 - Live 0xe0021000
> unix 29244 136 - Live 0xe0028000
> 
> Software Environment:
> Debian etch firewall using pppoe and openswan
> Problem Description:
> 
> The kernel spews hundreds of 
> SKB BUG: Invalid truesize (560) len=392, sizeof(sk_buff)=176
> With various values for truesize and len whenever ipsec tunnels are up.
> 
> No other detectible problems, except log pollution.
> 
> Steps to reproduce:
> /etc/init.d/ipsec start
Comment 2 Tony Lill 2008-06-30 14:59:16 UTC
Andrew Morton <akpm@linux-foundation.org> writes:

> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
>
> On Fri, 27 Jun 2008 16:37:22 -0700 (PDT)
> bugme-daemon@bugzilla.kernel.org wrote:
>
>> http://bugzilla.kernel.org/show_bug.cgi?id=10996
>> 
>>            Summary: 2.6.25 kernels spew SKB BUG: Invalid truesize when using
>>                     ipsec/openswan
>>            Product: Networking
>>            Version: 2.5
>>      KernelVersion: Linux version 2.6.25.9 (ajlill@ds) (gcc version 3.4.6
>>                     (Debian 3.
>>           Platform: All
>>         OS/Version: Linux
>>               Tree: Mainline
>>             Status: NEW
>>           Severity: low
>>           Priority: P1
>>          Component: IPV4
>>         AssignedTo: shemminger@linux-foundation.org
>>         ReportedBy: ajlill@ajlc.waterloo.on.ca
>> 
>> 
>> Latest working kernel version: 2.6.24.4
>> Earliest failing kernel version:2.6.25.4
>
> These two lines are contradictory.  Please clarify.

There were no messages when I was running 2.6.24.4, and there were
when I switched to 2.6.25.4. Now I haven't tried any kernels between
those, but I suspect it's a problem with the entire 2.6.25 branch.
Comment 3 Christian Felsing 2008-07-10 23:40:28 UTC
Can confirm this on an Intel Q965 mainboard on kernel 2.6.25.4 and 2.6.25.10:

# cat /proc/modules
i2c_dev 11472 0 - Live 0xffffffff88245000
authenc 9856 2 - Live 0xffffffff88241000
xfrm4_mode_tunnel 5440 4 - Live 0xffffffff8823e000
sit 16360 0 - Live 0xffffffff88239000
ip_gre 18208 0 - Live 0xffffffff88233000
dummy 6080 0 - Live 0xffffffff88230000
af_packet 28992 2 - Live 0xffffffff88227000
fan 8584 0 - Live 0xffffffff88223000
ac 9032 0 - Live 0xffffffff8821f000
battery 17544 0 - Live 0xffffffff88219000
nf_conntrack_ipv6 22408 1 - Live 0xffffffff88212000
ip6table_filter 5888 1 - Live 0xffffffff8820f000
ip6_tables 26768 1 ip6table_filter, Live 0xffffffff88207000
ipv6 344904 94 sit,nf_conntrack_ipv6, Live 0xffffffff881b1000
xt_tcpudp 5888 47 - Live 0xffffffff881ae000
xt_state 4992 3 - Live 0xffffffff881ab000
ipt_LOG 9156 4 - Live 0xffffffff881a7000
xt_limit 5636 4 - Live 0xffffffff881a4000
iptable_filter 6144 1 - Live 0xffffffff881a1000
xfrm_user 33024 2 - Live 0xffffffff88197000
iptable_nat 10576 1 - Live 0xffffffff88193000
nf_nat 25816 1 iptable_nat, Live 0xffffffff8818b000
nf_conntrack_ipv4 22864 5 iptable_nat,nf_nat, Live 0xffffffff88184000
xfrm4_tunnel 5440 0 - Live 0xffffffff88181000
tunnel4 6608 2 sit,xfrm4_tunnel, Live 0xffffffff8817e000
nf_conntrack 80144 5 nf_conntrack_ipv6,xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4, Live 0xffffffff88169000
ipcomp 11468 0 - Live 0xffffffff88165000
ip_tables 25104 2 iptable_filter,iptable_nat, Live 0xffffffff8815d000
esp4 11456 2 - Live 0xffffffff88159000
aead 12032 2 authenc,esp4, Live 0xffffffff88155000
ah4 9536 0 - Live 0xffffffff88151000
x_tables 28424 7 ip6_tables,xt_tcpudp,xt_state,ipt_LOG,xt_limit,iptable_nat,ip_tables, Live 0xffffffff88149000
deflate 6592 0 - Live 0xffffffff88146000
ctr 8128 0 - Live 0xffffffff88143000
twofish 8704 0 - Live 0xffffffff8813f000
twofish_common 17216 1 twofish, Live 0xffffffff88139000
camellia 20928 0 - Live 0xffffffff88132000
serpent 19968 0 - Live 0xffffffff8812c000
blowfish 10688 0 - Live 0xffffffff88128000
des_generic 19328 0 - Live 0xffffffff88122000
aes_x86_64 10816 4 - Live 0xffffffff8811e000
aes_generic 29928 1 aes_x86_64, Live 0xffffffff88115000
xcbc 8520 0 - Live 0xffffffff88111000
sha256_generic 11840 0 - Live 0xffffffff8810d000
crypto_null 6144 0 - Live 0xffffffff8810a000
hmac 7680 2 - Live 0xffffffff88107000
crypto_hash 4928 3 authenc,xcbc,hmac, Live 0xffffffff88104000
af_key 47384 0 - Live 0xffffffff880f7000
eeprom 10576 0 - Live 0xffffffff880f3000
ppp_deflate 8896 0 - Live 0xffffffff880ef000
zlib_deflate 24600 2 deflate,ppp_deflate, Live 0xffffffff880e7000
zlib_inflate 22080 2 deflate,ppp_deflate, Live 0xffffffff880e0000
bsd_comp 8640 0 - Live 0xffffffff880dc000
sha1_generic 5056 2 - Live 0xffffffff880d9000
arc4 4416 0 - Live 0xffffffff880d6000
ecb 6208 0 - Live 0xffffffff880d3000
ppp_mppe 10248 0 - Live 0xffffffff880cf000
ppp_generic 35048 3 ppp_deflate,bsd_comp,ppp_mppe, Live 0xffffffff880c5000
slhc 8960 1 ppp_generic, Live 0xffffffff880c1000
loop 23172 0 - Live 0xffffffff880ba000
usbhid 36896 0 - Live 0xffffffff880af000
8250_pnp 14400 0 - Live 0xffffffff880aa000
8250_pci 30720 0 - Live 0xffffffff880a1000
iTCO_wdt 16528 0 - Live 0xffffffff8809b000
i2c_i801 13404 0 - Live 0xffffffff88094000
8250 53832 2 8250_pnp,8250_pci, Live 0xffffffff88085000
iTCO_vendor_support 6660 1 iTCO_wdt, Live 0xffffffff88082000
serial_core 28736 1 8250, Live 0xffffffff88079000
thermal 24352 0 - Live 0xffffffff88070000
button 11680 0 - Live 0xffffffff8806a000
processor 38192 1 thermal, Live 0xffffffff8805d000
e1000 140288 0 - Live 0xffffffff88037000
i2c_core 30688 3 i2c_dev,eeprom,i2c_i801, Live 0xffffffff8802e000
uhci_hcd 30552 0 - Live 0xffffffff88025000
ehci_hcd 42764 0 - Live 0xffffffff88017000
rtc 19136 0 - Live 0xffffffff8800f000
evdev 16896 0 - Live 0xffffffff88007000
serio_raw 9988 0 - Live 0xffffffff88003000
pcspkr 5760 0 - Live 0xffffffff88000000

# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 CPU          6700  @ 2.66GHz
stepping        : 6
cpu MHz         : 2664.016
cache size      : 4096 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips        : 5330.14
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 CPU          6700  @ 2.66GHz
stepping        : 6
cpu MHz         : 2664.016
cache size      : 4096 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips        : 5328.06
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

# uname -a
                                                                                             Linux matrix 2.6.25.10 #1 SMP PREEMPT Thu Jul 10 13:18:09 CEST 2008 x86_64 GNU/Linux

Debian Etch with latest updates.

# lspci
00:00.0 Host bridge: Intel Corporation Q963/Q965 Memory Controller Hub (rev 02)
00:02.0 VGA compatible controller: Intel Corporation Q963/Q965 Integrated Graphics Controller (rev 02)
00:03.0 Communication controller: Intel Corporation Q963/Q965 HECI Controller (rev 02)
00:03.2 IDE interface: Intel Corporation Q963/Q965 PT IDER Controller (rev 02)
00:03.3 Serial controller: Intel Corporation Q963/Q965 KT Controller (rev 02)
00:19.0 Ethernet controller: Intel Corporation 82566DM Gigabit Network Connection (rev 02)
00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #4 (rev 02)
00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #5 (rev 02)
00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI #2 (rev 02)
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 02)
00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 02)
00:1c.1 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 2 (rev 02)
00:1c.2 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 3 (rev 02)
00:1c.3 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 4 (rev 02)
00:1c.4 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 5 (rev 02)
00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #3 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI #1 (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev f2)
00:1f.0 ISA bridge: Intel Corporation 82801HO (ICH8DO) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801HB (ICH8) SATA AHCI Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801H (ICH8 Family) SMBus Controller (rev 02)
02:00.0 IDE interface: Marvell Technology Group Ltd. Unknown device 6101 (rev b1)
06:03.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link)

seems problem depends on hardware, because same config on other hardware does not yield into this problem. Following HW is used with 2.6.25.4 and OpenSwan, also - but there are no SKB msgs:

# lspci
00:00.0 Host bridge: Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller (rev 03)
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 02)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 02)
00:1c.1 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 2 (rev 02)
00:1c.2 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 3 (rev 02)
00:1c.3 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 4 (rev 02)
00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 02)
00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev e2)
00:1f.0 ISA bridge: Intel Corporation 82801GBM (ICH7-M) LPC Interface Bridge (rev 02)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 02)
00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA IDE Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 02)
01:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller
02:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller
03:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller
Comment 4 Trevor Cordes 2008-07-14 09:25:48 UTC
This bug just hit me today and it seems to have completely locked out all network access to the box requiring a reset button push (headless box).

Jul 13 10:44:20 firewall kernel: SKB BUG: Invalid truesize (304) len=136, sizeof(sk_buff)=176

Looking at the logs, I see that it's popped up a few times before this but did not lead to any perceptible detrimental symptoms.

We ran 2.6.25.4-10.fc8 for 16 days without any SKB BUG log (and 2.6.24 before that without it also).  Then we rebooted into 2.6.25.6-27.fc8 and in 3 days the bug started showing up.  Looks like the bug is post-2.6.25.4-10.fc8, unless 16 days wasn't enough time for it to trigger.

We are also running PPPoE and ipsec.  We are NOT running openswan or raccoon, just a manually setup ipsec.conf and 2.6sec.

I have a hunch this bug is related to the PPPoE "waiting for pppX to become free" bug which I've fallen behind on following.  A quick check of the logs shows that the "waiting" bug may be fixed, but perhaps this bug has been substituted in its place (same symptoms, loss of all network connectivity).

The last kernel log messages before reset button (so some may have been lost) are:

Jul 14 11:00:04 firewall kernel: SKB BUG: Invalid truesize (304) len=136, sizeof(sk_buff)=176
Jul 14 11:00:15 firewall kernel:last message repeated 281 times
Jul 14 11:00:15 firewall bcrelay[13431]: ignored ENOBUFS from sendto(), temporary shortage of buffer memory
Jul 14 11:00:15 firewall bcrelay[13431]:last message repeated 14 times
Jul 14 11:00:15 firewall kernel: SKB BUG: Invalid truesize (304) len=136, sizeof(sk_buff)=176
Comment 5 Christian Felsing 2008-08-01 22:01:22 UTC
Bug persists also in kernel 2.6.25.14, but no impact in system stability.
Comment 6 Torsten Krah 2008-09-25 09:38:44 UTC
I can confirm this one.
Updating from 2.6.23.12 -> 2.6.26.5 i got this one too.
Its a old athlon 800, running ipsec with racoon.
Comment 7 Michal/ Mirosl/aw 2008-11-10 07:00:59 UTC
I just got a bit of those on kernel 2.6.27.5 on sparc64.

SKB BUG: Invalid truesize (288) len=66, sizeof(sk_buff)=224
SKB BUG: Invalid truesize (736) len=515, sizeof(sk_buff)=224

Occurred around the time I restarted the other end of manually configured IPsec tunnel. No other kernel messages showed up. The system looks stable and working.
Comment 8 Krzysztof Oledzki 2008-11-11 06:06:37 UTC
/me too

Started after upgrading 2.6.24 -> 2.6.26
Comment 9 Mariusz Drozdziel 2008-11-19 07:49:19 UTC
Appers on every box running IPSEC (racoon/openswan) and OpenVPN with kernel 2.6.26.
Comment 10 Christian Felsing 2008-12-29 01:36:31 UTC
Problem escalation since 2.6.28:

Dec 29 10:25:33 corebb kernel: ------------[ cut here ]------------
Dec 29 10:25:33 corebb kernel: WARNING: at net/core/skbuff.c:154 sock_rfree+0x64/0x70()
Dec 29 10:25:33 corebb kernel: SKB BUG: Invalid truesize (360) len=150, sizeof(sk_buff)=232
Dec 29 10:25:33 corebb kernel: Modules linked in: authenc xfrm4_mode_tunnel ip_gre dummy af_packet cpufreq_ondemand ac battery sg sr_mod cdrom nf_conntrack_ipv6 ip6t_rt ip6t_REJECT ip6table_filter ip6_tables ipv6 xt_tcpudp xt_state ipt_LOG xt_limit ipt_REJECT iptable_filter iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 deflate ctr twofish twofish_common camellia serpent blowfish des_generic aes_x86_64 aes_generic xcbc rmd160 sha256_generic hmac crypto_null af_key it87 w83791d hwmon_vid hwmon eeprom ppp_deflate zlib_deflate zlib_inflate bsd_comp sha1_generic arc4 ecb ppp_mppe ppp_generic slhc powernow_k8 loop usb_storage fan thermal button processor thermal_sys forcedeth serio_raw ohci_hcd ehci_hcd pcspkr evdev i2c_nforce2 i2c_core
Dec 29 10:25:33 corebb kernel: Pid: 2741, comm: snort Tainted: G        W  2.6.28 #1
Dec 29 10:25:33 corebb kernel: Call Trace:
Dec 29 10:25:33 corebb kernel: [<ffffffff8023a6ad>] warn_slowpath+0xcd/0x110
Dec 29 10:25:33 corebb kernel: [<ffffffff80236baa>] dequeue_task_fair+0xca/0xf0
Dec 29 10:25:33 corebb kernel: [<ffffffff80230991>] update_curr+0x51/0xd0
Dec 29 10:25:33 corebb kernel: [<ffffffff80234d8b>] finish_task_switch+0x3b/0xc0
Dec 29 10:25:33 corebb kernel: [<ffffffff8056d504>] thread_return+0x41/0x50d
Dec 29 10:25:33 corebb kernel: [<ffffffff80530590>] udp_poll+0x0/0x120
Dec 29 10:25:33 corebb kernel: [<ffffffff803dd914>] __ratelimit+0x4/0xe0
Dec 29 10:25:33 corebb kernel: [<ffffffff803dd9cb>] __ratelimit+0xbb/0xe0
Dec 29 10:25:33 corebb kernel: [<ffffffff804dbbc4>] sock_rfree+0x64/0x70
Dec 29 10:25:33 corebb kernel: [<ffffffff804df59e>] skb_release_head_state+0x4e/0xe0
Dec 29 10:25:33 corebb kernel: [<ffffffff804dfdf9>] skb_release_all+0x9/0x20
Dec 29 10:25:33 corebb kernel: [<ffffffff804df469>] __kfree_skb+0x9/0x80
Dec 29 10:25:33 corebb kernel: [<ffffffff804e24ac>] skb_free_datagram+0xc/0x40
Dec 29 10:25:33 corebb kernel: [<ffffffffa030a993>] packet_recvmsg+0x1b3/0x1e0 [af_packet]
Dec 29 10:25:33 corebb kernel: [<ffffffff804d82ea>] sock_recvmsg+0x14a/0x160
Dec 29 10:25:33 corebb kernel: [<ffffffff8024fc10>] autoremove_wake_function+0x0/0x30
Dec 29 10:25:33 corebb kernel: [<ffffffff8023f1b9>] ns_to_timeval+0x9/0x40
Dec 29 10:25:33 corebb kernel: [<ffffffff804d7bb1>] sockfd_lookup_light+0x41/0x80
Dec 29 10:25:33 corebb kernel: [<ffffffff804d99c2>] sys_recvfrom+0xe2/0x190
Dec 29 10:25:33 corebb kernel: [<ffffffff804da2f4>] sock_ioctl+0x54/0x240
Dec 29 10:25:33 corebb kernel: [<ffffffff802b426f>] vfs_ioctl+0x2f/0xa0
Dec 29 10:25:33 corebb kernel: [<ffffffff802b435c>] do_vfs_ioctl+0x7c/0x480
Dec 29 10:25:33 corebb kernel: [<ffffffff8020c20b>] system_call_fastpath+0x16/0x1b
Dec 29 10:25:33 corebb kernel: ---[ end trace e92f2c8f5a1520e3 ]---
Dec 29 10:25:42 corebb kernel: hmac crypto_null af_key it87 w83791d hwmon_vid hwmon eeprom ppp_deflate zlib_deflate zlib_inflate bsd_comp sha1_generic arc4 ecb ppp_mppe ppp_generic slhc powernow_k8 loop usb_storage fan thermal button processor thermal_sys forcedeth serio_raw ohci_hcd ehci_hcd pcspkr evdev i2c_nforce2 i2c_coreaarm0c04
Dec 29 10:25:42 corebb kernel: 4 <ffff0093]pce_evs+xb/xe a_akt
Dec 29 10:25:42 corebb kernel: 4 <ffff0d2a]sc_evs+x4/x6
Dec 29 10:25:42 corebb kernel: 4 <ffff04c0]atrmv_aefnto+x/x0<>[ffff8320> a_ieicl0001
Dec 29 10:25:42 corebb kernel: 4 <ffff0319]n_otmvl0904
Dec 29 10:25:42 corebb kernel: 4 <ffff0db1]scf_okplgt04/x0<>[ffff849c> y_evrm0e/x9
Dec 29 10:25:42 corebb kernel: 4 <ffff0d24]sc_ot+x4020<>[ffff8246> f_ot+xf0a
Dec 29 10:25:42 corebb kernel: 4 <ffff0b3c]d_f_ot+xc040<>[ffff8320> a_ieicl0001
Dec 29 10:25:42 corebb kernel: 4 <ffff002b]sse_alfspt+x601

This happens also on NVidia chipset mainboards. Besides spamming into logfile there seems no impact into system. IPsec is running w/o problems.

My recommendation:

1st: If this is a kernel problem, it should be fixed
2nd: If this is not a problem, there should be a method to turn off that syslog spamming ;-)
Comment 11 Thomas Szteliga 2009-01-09 03:23:32 UTC
I see this bug on my laptop (working as a router) and every time I see it, the my ethernet iface connected via USB

ID 05e9:0008 Kawasaki LSI KL5KUSB101B Ethernet [klsi] (working with kaweth.o)

"dies". The other iface (eth0: RealTek RTL-8169) works properly, even when the usb iface dies after "SKB BUG" log entries. I think this is not hardware/related, because i've seen and noticed the same when i was using a pcmcia WiFi card in place of "Kawasaki LSI KL5KUSB101B".

I'm NOT USING IPSEC, I use OpenVPN.

The only way to fix this for me is to do: `ifconfig eth1 down ; ifconfig eth1 up` and everything works again.

I've added this to my crontab, that helps me to keep my router working
with this bug:

*/1 * * * *  if ! ping -b -c 1 -w 2 -t 1 -I wlan0 192.168.66.253 1>/dev/null; then ifconfig eth1 down ; sleep 2; ifconfig eth1 up; fi

Where 192.168.66.253 is a host that works in my internal network and that should respond.


# uname -a
Linux aristo2600 2.6.27.6-router #1 Wed Nov 19 23:13:55 CET 2008 i686 Intel(R) Celeron(R) CPU 2.40GHz GenuineIntel GNU/Linux


# lspci
00:00.0 Host bridge: Silicon Integrated Systems [SiS] 650/M650 Host (rev 80)
00:01.0 PCI bridge: Silicon Integrated Systems [SiS] Virtual PCI-to-PCI bridge (AGP)
00:02.0 ISA bridge: Silicon Integrated Systems [SiS] SiS962 [MuTIOL Media IO] (rev 14)
00:02.1 SMBus: Silicon Integrated Systems [SiS] SiS961/2 SMBus Controller
00:02.3 FireWire (IEEE 1394): Silicon Integrated Systems [SiS] FireWire Controller
00:02.5 IDE interface: Silicon Integrated Systems [SiS] 5513 [IDE]
00:02.6 Modem: Silicon Integrated Systems [SiS] AC'97 Modem Controller (rev a0)
00:02.7 Multimedia audio controller: Silicon Integrated Systems [SiS] AC'97 Sound Controller (rev a0)
00:03.0 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f)
00:03.1 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f)
00:03.2 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f)
00:03.3 USB Controller: Silicon Integrated Systems [SiS] USB 2.0 Controller
00:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
00:0c.0 CardBus bridge: ENE Technology Inc CB1410 Cardbus Controller (rev 01)
01:00.0 VGA compatible controller: Silicon Integrated Systems [SiS] 65x/M650/740 PCI/AGP VGA Display Adapter


# lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 008: ID 05e9:0008 Kawasaki LSI KL5KUSB101B Ethernet [klsi]
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub


# lsmod
Module                  Size  Used by
sch_sfq                 6144  0
cls_u32                 6656  0
sch_htb                15488  0
iptable_mangle          2688  0
tun                    10116  2
xt_tcpudp               2816  14
xt_limit                2048  2
xt_state                1920  9
ipt_REJECT              2944  1
ipt_LOG                 5504  3
iptable_nat             5128  1
nf_nat                 15120  1 iptable_nat
nf_conntrack_ipv4      12812  12 iptable_nat,nf_nat
nf_conntrack           49696  4 xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4
iptable_filter          2560  1
ip_tables              10256  3 iptable_mangle,iptable_nat,iptable_filter
x_tables               12164  7 xt_tcpudp,xt_limit,xt_state,ipt_REJECT,ipt_LOG,iptable_nat,ip_tables
snd_pcsp                8836  0
snd_pcm                64392  1 snd_pcsp
rtc_cmos                9376  0
rtc_core               14872  1 rtc_cmos
snd_timer              18948  1 snd_pcm
rtc_lib                 2432  1 rtc_core
floppy                 52356  0
snd                    45988  3 snd_pcsp,snd_pcm,snd_timer
soundcore               5472  1 snd
snd_page_alloc          7304  1 snd_pcm
psmouse                35984  0
serio_raw               5124  0
8139too                22272  0
yenta_socket           22796  0
mii                     4480  1 8139too
rsrc_nonstatic         10368  1 yenta_socket
pcmcia_core            30992  2 yenta_socket,rsrc_nonstatic
8250_pnp               13952  0
8250_pci               22784  0
kaweth                 12288  0
nsc_ircc               15632  0
irda                  100536  1 nsc_ircc
crc_ccitt               1792  1 irda
ide_cd_mod             30596  0
8250                   21620  2 8250_pnp,8250_pci
serial_core            18304  1 8250
container               3328  0
firewire_ohci          20740  0
firewire_core          35608  1 firewire_ohci
ohci1394               28208  0
ieee1394               77252  1 ohci1394
battery                 9988  0
ac                      3972  0
thermal                15004  0
button                  5904  0
processor              27952  1 thermal
thermal_sys            11040  2 thermal,processor
hwmon                   2196  1 thermal_sys
i2c_sis96x              4228  0
shpchp                 28692  0
pci_hotplug            12296  1 shpchp
Comment 12 Krzysztof Oledzki 2009-01-09 03:28:24 UTC
(In reply to comment #11)
> I see this bug on my laptop (working as a router) and every time I see it,
> the
> my ethernet iface connected via USB
> 
> ID 05e9:0008 Kawasaki LSI KL5KUSB101B Ethernet [klsi] (working with kaweth.o)
> 
> "dies". The other iface (eth0: RealTek RTL-8169) works properly, even when
> the
> usb iface dies after "SKB BUG" log entries. I think this is not
> hardware/related, because i've seen and noticed the same when i was using a
> pcmcia WiFi card in place of "Kawasaki LSI KL5KUSB101B".
> 
> I'm NOT USING IPSEC, I use OpenVPN.

So this is a different bug, probably in kaweth. Please consider creating separate bugreport.
Comment 13 Christian Felsing 2009-02-10 00:54:25 UTC
Bug still exists on Kernel 2.6.28.4, can be verified on different HW platforms, at least on Intel Core2 and AMD FX62. Following output from AMD/NVidia mainboard.

# uname -a
Linux corebb 2.6.28.4 #1 SMP PREEMPT Sun Feb 8 11:12:25 CET 2009 x86_64 GNU/Linux

# lspci
00:00.0 RAM memory: nVidia Corporation C51 Host Bridge (rev a2)
00:00.1 RAM memory: nVidia Corporation C51 Memory Controller 0 (rev a2)
00:00.2 RAM memory: nVidia Corporation C51 Memory Controller 1 (rev a2)
00:00.3 RAM memory: nVidia Corporation C51 Memory Controller 5 (rev a2)
00:00.4 RAM memory: nVidia Corporation C51 Memory Controller 4 (rev a2)
00:00.5 RAM memory: nVidia Corporation C51 Host Bridge (rev a2)
00:00.6 RAM memory: nVidia Corporation C51 Memory Controller 3 (rev a2)
00:00.7 RAM memory: nVidia Corporation C51 Memory Controller 2 (rev a2)
00:04.0 PCI bridge: nVidia Corporation C51 PCI Express Bridge (rev a1)
00:08.0 RAM memory: nVidia Corporation MCP55 Memory Controller (rev a1)
00:09.0 ISA bridge: nVidia Corporation MCP55 LPC Bridge (rev a2)
00:09.1 SMBus: nVidia Corporation MCP55 SMBus (rev a2)
00:0a.0 USB Controller: nVidia Corporation MCP55 USB Controller (rev a1)
00:0a.1 USB Controller: nVidia Corporation MCP55 USB Controller (rev a2)
00:0c.0 IDE interface: nVidia Corporation MCP55 IDE (rev a1)
00:0d.0 IDE interface: nVidia Corporation MCP55 SATA Controller (rev a2)
00:0d.1 IDE interface: nVidia Corporation MCP55 SATA Controller (rev a2)
00:0d.2 IDE interface: nVidia Corporation MCP55 SATA Controller (rev a2)
00:0e.0 PCI bridge: nVidia Corporation MCP55 PCI bridge (rev a2)
00:10.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
00:11.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
00:12.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev a2)
00:16.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev a2)
00:17.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev a2)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
01:00.0 VGA compatible controller: nVidia Corporation GeForce 7300 GS (rev a1)
02:0b.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link)
04:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01)
corebb:/usr/src# lsusb
Bus 002 Device 002: ID 14dd:0002
Bus 002 Device 001: ID 1d6b:0002
Bus 001 Device 001: ID 1d6b:0001

# lsmod
Module                  Size  Used by
authenc                 8448  12
xfrm4_mode_tunnel       4928  24
xfrm_user              26560  2
xfrm4_tunnel            4928  0
af_key                 35224  0
ip_gre                 18372  0
dummy                   5120  0
af_packet              22336  2
cpufreq_ondemand        9872  0
ac                      7240  0
battery                14856  0
nf_conntrack_ipv6      17480  1
ip6t_rt                 5056  3
ip6t_REJECT             6528  4
ip6table_filter         5312  1
ip6_tables             22416  2 ip6t_rt,ip6table_filter
ipv6                  327024  46 nf_conntrack_ipv6,ip6t_REJECT
xt_tcpudp               5696  53
xt_state                4544  3
ipt_LOG                 8516  5
xt_limit                4996  4
ipt_REJECT              5696  4
iptable_filter          5440  1
iptable_nat             8516  1
nf_nat                 23828  1 iptable_nat
nf_conntrack_ipv4      18072  5 iptable_nat,nf_nat
nf_conntrack           76624  5 nf_conntrack_ipv6,xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4
nf_defrag_ipv4          4480  1 nf_conntrack_ipv4
ip_tables              20816  2 iptable_filter,iptable_nat
x_tables               25480  10 ip6t_rt,ip6t_REJECT,ip6_tables,xt_tcpudp,xt_state,ipt_LOG,xt_limit,ipt_REJECT,iptable_nat,ip_tables
tunnel4                 5968  1 xfrm4_tunnel
ipcomp                  5632  0
xfrm_ipcomp             8396  1 ipcomp
esp4                    9216  12
ah4                     7744  0
deflate                 5440  0
ctr                     6976  0
twofish                 8576  0
twofish_common         16704  1 twofish
camellia               21312  0
serpent                21376  0
blowfish               10816  0
des_generic            19392  0
aes_x86_64             10368  12
aes_generic            30568  1 aes_x86_64
xcbc                    7560  0
rmd160                 10048  0
sha256_generic         11712  0
hmac                    6784  12
crypto_null             5696  0
sg                     34656  0
sr_mod                 18948  0
cdrom                  40104  1 sr_mod
it87                   28888  0
w83791d                27228  0
hwmon_vid               5888  2 it87,w83791d
hwmon                   5256  2 it87,w83791d
eeprom                  8720  0
ppp_deflate             7424  0
zlib_deflate           23448  2 deflate,ppp_deflate
zlib_inflate           18432  2 deflate,ppp_deflate
bsd_comp                7872  0
sha1_generic            4928  12
arc4                    4160  0
ecb                     5312  0
ppp_mppe                9352  0
ppp_generic            28584  3 ppp_deflate,bsd_comp,ppp_mppe
slhc                    8896  1 ppp_generic
powernow_k8            16068  1
loop                   19020  0
usb_storage            97472  0
fan                     7496  0
thermal                20896  0
i2c_nforce2             9664  0
button                  9632  0
processor              48184  2 powernow_k8,thermal
thermal_sys            14464  3 fan,thermal,processor
serio_raw               8260  0
forcedeth              60944  0
ehci_hcd               39052  0
ohci_hcd               27100  0
i2c_core               29664  3 w83791d,eeprom,i2c_nforce2
pcspkr                  5120  0
evdev                  13856  0


[  309.701229] ------------[ cut here ]------------
[  309.701271] WARNING: at net/core/skbuff.c:154 sock_rfree+0x64/0x70()
[  309.701312] SKB BUG: Invalid truesize (360) len=158, sizeof(sk_buff)=232
[  309.701352] Modules linked in: authenc xfrm4_mode_tunnel xfrm_user xfrm4_tunnel af_key ip_gre dummy af_packet cpufreq_ondemand ac battery nf_conntrack_ipv6 ip6t_rt ip6t_REJECT ip6table_filter ip6_tables ipv6 xt_tcpudp xt_state ipt_LOG xt_limit ipt_REJECT iptable_filter iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables tunnel4 ipcomp xfrm_ipcomp esp4 ah4 deflate ctr twofish twofish_common camellia serpent blowfish des_generic aes_x86_64 aes_generic xcbc rmd160 sha256_generic hmac crypto_null sg sr_mod cdrom it87 w83791d hwmon_vid hwmon eeprom ppp_deflate zlib_deflate zlib_inflate bsd_comp sha1_generic arc4 ecb ppp_mppe ppp_generic slhc powernow_k8 loop usb_storage fan thermal i2c_nforce2 button processor thermal_sys serio_raw forcedeth ehci_hcd ohci_hcd i2c_core pcspkr evdev [last unloaded: xfrm_user]
[  309.703604] Pid: 2697, comm: snort Tainted: G        W  2.6.28.4 #1
[  309.703644] Call Trace:
[  309.703684]  [<ffffffff8023a73d>] warn_slowpath+0xcd/0x110
[  309.703724]  [<ffffffff80236c3a>] dequeue_task_fair+0xca/0xf0
[  309.703764]  [<ffffffff80230f7d>] __dequeue_entity+0x3d/0x50
[  309.703804]  [<ffffffff80230fb5>] set_next_entity+0x25/0x50
[  309.703844]  [<ffffffff8056d704>] thread_return+0x41/0x50d
[  309.703885]  [<ffffffff803a23b0>] cap_socket_recvmsg+0x0/0x10
[  309.703925]  [<ffffffff8056de85>] schedule_timeout+0x95/0xd0
[  309.703966]  [<ffffffff8024fdf3>] prepare_to_wait_exclusive+0x23/0x70
[  309.704012]  [<ffffffff804e24cb>] __skb_recv_datagram+0x1eb/0x280
[  309.704053]  [<ffffffff804e17b8>] memcpy_toiovec+0x8/0x70
[  309.704093]  [<ffffffff803dda4b>] __ratelimit+0xbb/0xe0
[  309.704140]  [<ffffffff804dbdb4>] sock_rfree+0x64/0x70
[  309.704185]  [<ffffffff804df78e>] skb_release_head_state+0x4e/0xe0
[  309.704247]  [<ffffffff804dffe9>] skb_release_all+0x9/0x20
[  309.704304]  [<ffffffff804df659>] __kfree_skb+0x9/0x80
[  309.704345]  [<ffffffff804e269c>] skb_free_datagram+0xc/0x40
[  309.704388]  [<ffffffffa030a993>] packet_recvmsg+0x1b3/0x1e0 [af_packet]
[  309.704429]  [<ffffffff804d84da>] sock_recvmsg+0x14a/0x160
[  309.704476]  [<ffffffff8024fce0>] autoremove_wake_function+0x0/0x30
[  309.704534]  [<ffffffff8023f259>] ns_to_timeval+0x9/0x40
[  309.704587]  [<ffffffff804d7da1>] sockfd_lookup_light+0x41/0x80
[  309.704633]  [<ffffffff804d9bb2>] sys_recvfrom+0xe2/0x190
[  309.704672]  [<ffffffff804da4e4>] sock_ioctl+0x54/0x240
[  309.704711]  [<ffffffff802b419f>] vfs_ioctl+0x2f/0xa0
[  309.704750]  [<ffffffff802b428c>] do_vfs_ioctl+0x7c/0x480
[  309.704801]  [<ffffffff8020c20b>] system_call_fastpath+0x16/0x1b
[  309.704860] ---[ end trace 358b71b4626cc696 ]---

Is there any way to turn off that logs if problem does not impact system performance / stability ? Turning off IPsec is not an option.
Comment 14 Tomasz Torcz 2009-02-18 05:48:21 UTC
What the status of this bug? I'm hitting it recently.
Linux anthrax 2.6.27 #1 Fri Oct 10 17:16:44 CEST 2008 i586 Pentium MMX GenuineIntel GNU/Linux

It's pentium166mmx with 2x e100 and 1xe1000, working mainly as router (IPv4 ethernet/IPv6 aiccu tunel froms Sixxs). 160 MB of RAM, irssi, mldonkey, samba, ftp, maradns, apache, postgresql.

IPSec is selected in .config, but not used in anyway. The same for traffic control. Netfilter has about 2 dozen of rules.

This message appears often in logs, but no connection is broken and machine still works.

% dmesg -s 10000 | sort -u
SKB BUG: Invalid truesize (272) len=98, sizeof(sk_buff)=176
SKB BUG: Invalid truesize (304) len=134, sizeof(sk_buff)=176
SKB BUG: Invalid truesize (400) len=230, sizeof(sk_buff)=176

% dmesg -s 10000 | grep -c "SKB BUG"
156
Comment 15 Bernhard Schmidt 2009-03-15 07:28:54 UTC
I have the same problem with 2.6.28.7. 

[828090.107912] ------------[ cut here ]------------
[828090.161915] WARNING: at net/core/skbuff.c:150 sock_rfree+0x22/0x51()
[828090.235950] SKB BUG: Invalid truesize (360) len=134, sizeof(sk_buff)=232
[828090.313892] Modules linked in: tg3 esp6 xfrm6_mode_tunnel authenc esp4 xfrm4_mode_tunnel tun kvm_intel kvm xt_NOTRACK ip6table_raw ip6t_LOG nf_conntrack_ipv6 ip6table_filter ip6_tables xt_physdev ipt_LOG xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_hashlimit iptable_filter ip_tables x_tables bridge stp llc deflate zlib_deflate zlib_inflate ctr twofish twofish_common camellia serpent blowfish des_generic cbc aes_x86_64 aes_generic xcbc sha256_generic sha1_generic crypto_null af_key dm_crypt ipv6 coretemp loop ipmi_si ipmi_msghandler hpilo hpwdt pcspkr shpchp pci_hotplug container button psmouse serio_raw evdev ext3 jbd dm_mirror dm_region_hash dm_log dm_snapshot dm_mod raid1 md_mod sg sd_mod sr_mod cdrom usbhid hid ata_piix ata_generic libata scsi_mod ide_pci_generic ide_core ehci_hcd libphy uhci_hcd thermal processor fan thermal_sys [last unloaded: tg3]
[828091.208842] Pid: 25130, comm: tcpdump Tainted: G        W  2.6.28.7 #1
[828091.284891] Call Trace:
[828091.313855]  [<ffffffff802398aa>] warn_slowpath+0xb4/0xda
[828091.376854]  [<ffffffff80235a9a>] finish_task_switch+0x2b/0xc4
[828091.444884]  [<ffffffff8043885a>] thread_return+0x3d/0xc5
[828091.507894]  [<ffffffff80377169>] n_tty_receive_buf+0xd1e/0xd6b
[828091.576873]  [<ffffffff80243f63>] __dequeue_signal+0xfb/0x124
[828091.643860]  [<ffffffff802454c3>] dequeue_signal+0x9c/0x113
[828091.708918]  [<ffffffff804389e0>] schedule_timeout+0x1e/0xad
[828091.775878]  [<ffffffff80439e36>] _spin_lock_irq+0xd/0xf
[828091.837859]  [<ffffffff8020b8d6>] do_notify_resume+0x718/0x837
[828091.905859]  [<ffffffff80379445>] pty_write_room+0x0/0x1f
[828091.968877]  [<ffffffff80439e36>] _spin_lock_irq+0xd/0xf
[828092.030908]  [<ffffffff80438125>] wait_for_common+0x129/0x14d
[828092.097902]  [<ffffffff80221be9>] default_spin_lock_flags+0x5/0x8
[828092.168919]  [<ffffffff80439e21>] _spin_lock_irqsave+0x24/0x2c
[828092.236923]  [<ffffffff80335e51>] __ratelimit+0xb5/0xc0
[828092.297980]  [<ffffffff803c151d>] sock_rfree+0x22/0x51
[828092.357907]  [<ffffffff80439e21>] _spin_lock_irqsave+0x24/0x2c
[828092.425904]  [<ffffffff803c5056>] skb_release_head_state+0x60/0xbb
[828092.497888]  [<ffffffff803c476c>] __kfree_skb+0x9/0x77
[828092.557868]  [<ffffffff803c4809>] skb_queue_purge+0xb/0x1d
[828092.621887]  [<ffffffff80428d5f>] packet_release+0x16f/0x18f
[828092.687841]  [<ffffffff8043885a>] thread_return+0x3d/0xc5
[828092.750858]  [<ffffffff803bf06a>] sock_release+0x19/0x72
[828092.813834]  [<ffffffff803bf0e5>] sock_close+0x22/0x27
[828092.873839]  [<ffffffff8029f82d>] __fput+0xc6/0x18a
[828092.930840]  [<ffffffff8029d129>] filp_close+0x50/0x57
[828092.990823]  [<ffffffff8029d1cb>] sys_close+0x9b/0xdb
[828093.049845]  [<ffffffff8020be4a>] system_call_fastpath+0x16/0x1b
[828093.119856] ---[ end trace 444b547394c96982 ]---

I think it's related to IPsec here as well, I only hit this BUG when I run tcpdump on the external interface and it sees an ESP packet.