Bug 10996
Summary: | 2.6.25 kernels spew SKB BUG: Invalid truesize when using ipsec/openswan | ||
---|---|---|---|
Product: | Networking | Reporter: | Tony Lill (ajlill) |
Component: | IPV4 | Assignee: | Stephen Hemminger (stephen) |
Status: | CLOSED CODE_FIX | ||
Severity: | low | CC: | aaron, alan, berni, daniel.rode, hostmaster, jbohac, mdrozdziel, mishu, ole, zdzichu |
Priority: | P1 | ||
Hardware: | All | ||
OS: | Linux | ||
Kernel Version: | 2.6.25.9 | Subsystem: | |
Regression: | No | Bisected commit-id: |
Description
Tony Lill
2008-06-27 16:37:21 UTC
Reply-To: akpm@linux-foundation.org (switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Fri, 27 Jun 2008 16:37:22 -0700 (PDT) bugme-daemon@bugzilla.kernel.org wrote: > http://bugzilla.kernel.org/show_bug.cgi?id=10996 > > Summary: 2.6.25 kernels spew SKB BUG: Invalid truesize when using > ipsec/openswan > Product: Networking > Version: 2.5 > KernelVersion: Linux version 2.6.25.9 (ajlill@ds) (gcc version 3.4.6 > (Debian 3. > Platform: All > OS/Version: Linux > Tree: Mainline > Status: NEW > Severity: low > Priority: P1 > Component: IPV4 > AssignedTo: shemminger@linux-foundation.org > ReportedBy: ajlill@ajlc.waterloo.on.ca > > > Latest working kernel version: 2.6.24.4 > Earliest failing kernel version:2.6.25.4 These two lines are contradictory. Please clarify. > Distribution: Debian etch > Hardware Environment: > processor : 0 > vendor_id : GenuineIntel > cpu family : 15 > model : 4 > model name : Intel(R) Celeron(R) CPU 2.93GHz > stepping : 1 > cpu MHz : 2929.700 > cache size : 256 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 5 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca > cmov > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe constant_tsc up > pebs > bts pni monitor ds_cpl cid xtpr > bogomips : 5865.13 > clflush size : 64 > > Linux-i686-2.6.25.9 $ cat /proc/modules > xfrm_user 28040 2 - Live 0xe037d000 > xfrm4_tunnel 3976 0 - Live 0xe0226000 > af_key 39452 0 - Live 0xe0308000 > authenc 8072 5 - Live 0xe0443000 > xfrm4_mode_tunnel 3976 10 - Live 0xe0435000 > cls_u32 9100 1 - Live 0xe045e000 > sch_red 7432 1 - Live 0xe0440000 > xt_mark 3208 2 - Live 0xe0433000 > xt_layer7 12564 5 - Live 0xe0463000 > xt_length 2952 2 - Live 0xe0431000 > cls_fw 6536 14 - Live 0xe042b000 > sch_sfq 8072 14 - Live 0xe0422000 > sch_htb 17288 2 - Live 0xe0425000 > xt_TCPMSS 6536 1 - Live 0xe0400000 > xt_tcpmss 3208 1 - Live 0xe0418000 > netconsole 11044 0 - Live 0xe043c000 > configfs 28592 2 netconsole, Live 0xe0446000 > pppoe 14564 2 - Live 0xe0437000 > pppox 5396 1 pppoe, Live 0xe042e000 > iptable_mangle 4360 1 - Live 0xe0415000 > xt_MARK 3592 47 - Live 0xe0395000 > xt_conntrack 5128 1 - Live 0xe0412000 > ipt_REDIRECT 3208 1 - Live 0xe035f000 > xt_limit 3596 4 - Live 0xe0334000 > ipt_recent 10428 3 - Live 0xe041a000 > ipt_REJECT 5000 11 - Live 0xe040f000 > xt_state 3464 10 - Live 0xe0306000 > xt_tcpudp 4360 203 - Live 0xe040c000 > ipt_LOG 7052 27 - Live 0xe03f7000 > iptable_filter 4492 1 - Live 0xe03fd000 > iptable_nat 8340 1 - Live 0xe0408000 > ip_tables 15256 3 iptable_mangle,iptable_filter,iptable_nat, Live 0xe0403000 > nf_nat_pptp 4872 0 - Live 0xe03fa000 > nf_conntrack_pptp 8076 1 nf_nat_pptp, Live 0xe03a2000 > nf_conntrack_proto_gre 7176 1 nf_conntrack_pptp, Live 0xe034b000 > nf_nat_proto_gre 3852 1 nf_nat_pptp, Live 0xe0254000 > nf_nat_sip 5512 0 - Live 0xe0392000 > nf_conntrack_sip 10148 1 nf_nat_sip, Live 0xe039e000 > nf_nat_ftp 4364 0 - Live 0xe038f000 > nf_nat 20532 6 > ipt_REDIRECT,iptable_nat,nf_nat_pptp,nf_nat_proto_gre,nf_nat_sip, > nf_nat_ftp, Live 0xe0397000 > nf_conntrack_ipv4 18716 19 iptable_nat,nf_nat, Live 0xe0377000 > nf_conntrack_ftp 10292 1 nf_nat_ftp, Live 0xe038b000 > nf_conntrack 64396 13 > xt_layer7,xt_conntrack,xt_state,iptable_nat,nf_nat_pptp,nf > > _conntrack_pptp,nf_conntrack_proto_gre,nf_nat_sip,nf_conntrack_sip,nf_nat_ftp,nf > _nat,nf_conntrack_ipv4,nf_conntrack_ftp, Live 0xe03e6000 > xt_TARPIT 5000 2 - Live 0xe0374000 > x_tables 17804 17 > xt_mark,xt_layer7,xt_length,xt_TCPMSS,xt_tcpmss,xt_MARK,xt_con > > ntrack,ipt_REDIRECT,xt_limit,ipt_recent,ipt_REJECT,xt_state,xt_tcpudp,ipt_LOG,ip > table_nat,ip_tables,xt_TARPIT, Live 0xe0385000 > tunnel4 4752 1 xfrm4_tunnel, Live 0xe034e000 > ipcomp 9744 0 - Live 0xe0370000 > esp4 9096 5 - Live 0xe035b000 > aead 9096 2 authenc,esp4, Live 0xe0357000 > ah4 7432 0 - Live 0xe0341000 > af_packet 23176 8 - Live 0xe0369000 > ppp_generic 28196 6 pppoe,pppox, Live 0xe0361000 > slhc 7176 1 ppp_generic, Live 0xe004c000 > ipv6 261772 34 - Live 0xe03a5000 > deflate 4872 0 - Live 0xe0327000 > zlib_deflate 19776 1 deflate, Live 0xe0351000 > ctr 6024 0 - Live 0xe0316000 > twofish 8328 0 - Live 0xe0330000 > twofish_common 13832 1 twofish, Live 0xe033c000 > camellia 22536 0 - Live 0xe0344000 > serpent 18696 0 - Live 0xe0336000 > blowfish 10376 0 - Live 0xe0323000 > des_generic 17288 0 - Live 0xe032a000 > cbc 5384 5 - Live 0xe0313000 > aes_i586 9352 5 - Live 0xe0241000 > aes_generic 31052 1 aes_i586, Live 0xe031a000 > xcbc 6416 0 - Live 0xe023e000 > sha256_generic 12808 0 - Live 0xe0301000 > sha1_generic 3720 5 - Live 0xe017e000 > crypto_null 4232 0 - Live 0xe023b000 > crypto_blkcipher 19864 9 authenc,ctr,cbc,crypto_null, Live 0xe02fb000 > dm_snapshot 20400 0 - Live 0xe024e000 > dm_mirror 29200 0 - Live 0xe0245000 > dm_mod 63032 2 dm_snapshot,dm_mirror, Live 0xe026e000 > w83627hf 25140 0 - Live 0xe0164000 > hwmon_vid 3848 1 w83627hf, Live 0xe0052000 > hwmon 4388 1 w83627hf, Live 0xe0159000 > 8250_pnp 10888 0 - Live 0xe0237000 > snd_intel8x0 34252 0 - Live 0xe021c000 > snd_ac97_codec 93884 1 snd_intel8x0, Live 0xe0256000 > ac97_bus 2952 1 snd_ac97_codec, Live 0xe00a6000 > 8250 26776 1 8250_pnp, Live 0xe0214000 > serial_core 22536 1 8250, Live 0xe0177000 > floppy 57224 0 - Live 0xe0228000 > psmouse 39368 0 - Live 0xe016c000 > snd_pcm 73640 2 snd_intel8x0,snd_ac97_codec, Live 0xe0201000 > pcspkr 4232 0 - Live 0xe015c000 > rtc 15532 0 - Live 0xe013a000 > serio_raw 7948 0 - Live 0xe00f6000 > i2c_i801 10648 0 - Live 0xe013f000 > snd_timer 24600 1 snd_pcm, Live 0xe0143000 > i2c_core 24348 1 i2c_i801, Live 0xe010b000 > snd 52744 4 snd_intel8x0,snd_ac97_codec,snd_pcm,snd_timer, Live 0xe014b000 > soundcore 9188 1 snd, Live 0xe0136000 > snd_page_alloc 11412 2 snd_intel8x0,snd_pcm, Live 0xe0132000 > rng_core 6156 0 - Live 0xe001e000 > iTCO_wdt 12724 0 - Live 0xe00ed000 > button 9496 0 - Live 0xe00f2000 > intel_agp 27112 1 - Live 0xe0103000 > agpgart 35416 1 intel_agp, Live 0xe00f9000 > sg 36168 0 - Live 0xe00d4000 > evdev 13348 0 - Live 0xe00a1000 > sr_mod 17492 0 - Live 0xe00ce000 > cdrom 36292 1 sr_mod, Live 0xe008a000 > ext3 124888 3 - Live 0xe0112000 > jbd 50736 1 ext3, Live 0xe00df000 > mbcache 9740 1 ext3, Live 0xe0024000 > sd_mod 27296 10 - Live 0xe0094000 > pata_acpi 8200 0 - Live 0xe007e000 > ata_generic 9104 0 - Live 0xe007a000 > ata_piix 22796 10 - Live 0xe0083000 > libata 150576 3 pata_acpi,ata_generic,ata_piix, Live 0xe00a8000 > dock 12188 1 libata, Live 0xe0061000 > 8139cp 23440 0 - Live 0xe0073000 > 8139too 27544 0 - Live 0xe006b000 > mii 6408 2 8139cp,8139too, Live 0xe004f000 > ehci_hcd 34336 0 - Live 0xe0057000 > uhci_hcd 25244 0 - Live 0xe0031000 > thermal 21308 0 - Live 0xe0039000 > processor 44792 1 thermal, Live 0xe0040000 > fan 5256 0 - Live 0xe0021000 > unix 29244 136 - Live 0xe0028000 > > Software Environment: > Debian etch firewall using pppoe and openswan > Problem Description: > > The kernel spews hundreds of > SKB BUG: Invalid truesize (560) len=392, sizeof(sk_buff)=176 > With various values for truesize and len whenever ipsec tunnels are up. > > No other detectible problems, except log pollution. > > Steps to reproduce: > /etc/init.d/ipsec start Andrew Morton <akpm@linux-foundation.org> writes: > (switched to email. Please respond via emailed reply-to-all, not via the > bugzilla web interface). > > On Fri, 27 Jun 2008 16:37:22 -0700 (PDT) > bugme-daemon@bugzilla.kernel.org wrote: > >> http://bugzilla.kernel.org/show_bug.cgi?id=10996 >> >> Summary: 2.6.25 kernels spew SKB BUG: Invalid truesize when using >> ipsec/openswan >> Product: Networking >> Version: 2.5 >> KernelVersion: Linux version 2.6.25.9 (ajlill@ds) (gcc version 3.4.6 >> (Debian 3. >> Platform: All >> OS/Version: Linux >> Tree: Mainline >> Status: NEW >> Severity: low >> Priority: P1 >> Component: IPV4 >> AssignedTo: shemminger@linux-foundation.org >> ReportedBy: ajlill@ajlc.waterloo.on.ca >> >> >> Latest working kernel version: 2.6.24.4 >> Earliest failing kernel version:2.6.25.4 > > These two lines are contradictory. Please clarify. There were no messages when I was running 2.6.24.4, and there were when I switched to 2.6.25.4. Now I haven't tried any kernels between those, but I suspect it's a problem with the entire 2.6.25 branch. Can confirm this on an Intel Q965 mainboard on kernel 2.6.25.4 and 2.6.25.10: # cat /proc/modules i2c_dev 11472 0 - Live 0xffffffff88245000 authenc 9856 2 - Live 0xffffffff88241000 xfrm4_mode_tunnel 5440 4 - Live 0xffffffff8823e000 sit 16360 0 - Live 0xffffffff88239000 ip_gre 18208 0 - Live 0xffffffff88233000 dummy 6080 0 - Live 0xffffffff88230000 af_packet 28992 2 - Live 0xffffffff88227000 fan 8584 0 - Live 0xffffffff88223000 ac 9032 0 - Live 0xffffffff8821f000 battery 17544 0 - Live 0xffffffff88219000 nf_conntrack_ipv6 22408 1 - Live 0xffffffff88212000 ip6table_filter 5888 1 - Live 0xffffffff8820f000 ip6_tables 26768 1 ip6table_filter, Live 0xffffffff88207000 ipv6 344904 94 sit,nf_conntrack_ipv6, Live 0xffffffff881b1000 xt_tcpudp 5888 47 - Live 0xffffffff881ae000 xt_state 4992 3 - Live 0xffffffff881ab000 ipt_LOG 9156 4 - Live 0xffffffff881a7000 xt_limit 5636 4 - Live 0xffffffff881a4000 iptable_filter 6144 1 - Live 0xffffffff881a1000 xfrm_user 33024 2 - Live 0xffffffff88197000 iptable_nat 10576 1 - Live 0xffffffff88193000 nf_nat 25816 1 iptable_nat, Live 0xffffffff8818b000 nf_conntrack_ipv4 22864 5 iptable_nat,nf_nat, Live 0xffffffff88184000 xfrm4_tunnel 5440 0 - Live 0xffffffff88181000 tunnel4 6608 2 sit,xfrm4_tunnel, Live 0xffffffff8817e000 nf_conntrack 80144 5 nf_conntrack_ipv6,xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4, Live 0xffffffff88169000 ipcomp 11468 0 - Live 0xffffffff88165000 ip_tables 25104 2 iptable_filter,iptable_nat, Live 0xffffffff8815d000 esp4 11456 2 - Live 0xffffffff88159000 aead 12032 2 authenc,esp4, Live 0xffffffff88155000 ah4 9536 0 - Live 0xffffffff88151000 x_tables 28424 7 ip6_tables,xt_tcpudp,xt_state,ipt_LOG,xt_limit,iptable_nat,ip_tables, Live 0xffffffff88149000 deflate 6592 0 - Live 0xffffffff88146000 ctr 8128 0 - Live 0xffffffff88143000 twofish 8704 0 - Live 0xffffffff8813f000 twofish_common 17216 1 twofish, Live 0xffffffff88139000 camellia 20928 0 - Live 0xffffffff88132000 serpent 19968 0 - Live 0xffffffff8812c000 blowfish 10688 0 - Live 0xffffffff88128000 des_generic 19328 0 - Live 0xffffffff88122000 aes_x86_64 10816 4 - Live 0xffffffff8811e000 aes_generic 29928 1 aes_x86_64, Live 0xffffffff88115000 xcbc 8520 0 - Live 0xffffffff88111000 sha256_generic 11840 0 - Live 0xffffffff8810d000 crypto_null 6144 0 - Live 0xffffffff8810a000 hmac 7680 2 - Live 0xffffffff88107000 crypto_hash 4928 3 authenc,xcbc,hmac, Live 0xffffffff88104000 af_key 47384 0 - Live 0xffffffff880f7000 eeprom 10576 0 - Live 0xffffffff880f3000 ppp_deflate 8896 0 - Live 0xffffffff880ef000 zlib_deflate 24600 2 deflate,ppp_deflate, Live 0xffffffff880e7000 zlib_inflate 22080 2 deflate,ppp_deflate, Live 0xffffffff880e0000 bsd_comp 8640 0 - Live 0xffffffff880dc000 sha1_generic 5056 2 - Live 0xffffffff880d9000 arc4 4416 0 - Live 0xffffffff880d6000 ecb 6208 0 - Live 0xffffffff880d3000 ppp_mppe 10248 0 - Live 0xffffffff880cf000 ppp_generic 35048 3 ppp_deflate,bsd_comp,ppp_mppe, Live 0xffffffff880c5000 slhc 8960 1 ppp_generic, Live 0xffffffff880c1000 loop 23172 0 - Live 0xffffffff880ba000 usbhid 36896 0 - Live 0xffffffff880af000 8250_pnp 14400 0 - Live 0xffffffff880aa000 8250_pci 30720 0 - Live 0xffffffff880a1000 iTCO_wdt 16528 0 - Live 0xffffffff8809b000 i2c_i801 13404 0 - Live 0xffffffff88094000 8250 53832 2 8250_pnp,8250_pci, Live 0xffffffff88085000 iTCO_vendor_support 6660 1 iTCO_wdt, Live 0xffffffff88082000 serial_core 28736 1 8250, Live 0xffffffff88079000 thermal 24352 0 - Live 0xffffffff88070000 button 11680 0 - Live 0xffffffff8806a000 processor 38192 1 thermal, Live 0xffffffff8805d000 e1000 140288 0 - Live 0xffffffff88037000 i2c_core 30688 3 i2c_dev,eeprom,i2c_i801, Live 0xffffffff8802e000 uhci_hcd 30552 0 - Live 0xffffffff88025000 ehci_hcd 42764 0 - Live 0xffffffff88017000 rtc 19136 0 - Live 0xffffffff8800f000 evdev 16896 0 - Live 0xffffffff88007000 serio_raw 9988 0 - Live 0xffffffff88003000 pcspkr 5760 0 - Live 0xffffffff88000000 # cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz stepping : 6 cpu MHz : 2664.016 cache size : 4096 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm bogomips : 5330.14 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz stepping : 6 cpu MHz : 2664.016 cache size : 4096 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm bogomips : 5328.06 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: # uname -a Linux matrix 2.6.25.10 #1 SMP PREEMPT Thu Jul 10 13:18:09 CEST 2008 x86_64 GNU/Linux Debian Etch with latest updates. # lspci 00:00.0 Host bridge: Intel Corporation Q963/Q965 Memory Controller Hub (rev 02) 00:02.0 VGA compatible controller: Intel Corporation Q963/Q965 Integrated Graphics Controller (rev 02) 00:03.0 Communication controller: Intel Corporation Q963/Q965 HECI Controller (rev 02) 00:03.2 IDE interface: Intel Corporation Q963/Q965 PT IDER Controller (rev 02) 00:03.3 Serial controller: Intel Corporation Q963/Q965 KT Controller (rev 02) 00:19.0 Ethernet controller: Intel Corporation 82566DM Gigabit Network Connection (rev 02) 00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #4 (rev 02) 00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #5 (rev 02) 00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI #2 (rev 02) 00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 02) 00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 02) 00:1c.1 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 2 (rev 02) 00:1c.2 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 3 (rev 02) 00:1c.3 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 4 (rev 02) 00:1c.4 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 5 (rev 02) 00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #1 (rev 02) 00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #2 (rev 02) 00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI #3 (rev 02) 00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI #1 (rev 02) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev f2) 00:1f.0 ISA bridge: Intel Corporation 82801HO (ICH8DO) LPC Interface Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation 82801HB (ICH8) SATA AHCI Controller (rev 02) 00:1f.3 SMBus: Intel Corporation 82801H (ICH8 Family) SMBus Controller (rev 02) 02:00.0 IDE interface: Marvell Technology Group Ltd. Unknown device 6101 (rev b1) 06:03.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link) seems problem depends on hardware, because same config on other hardware does not yield into this problem. Following HW is used with 2.6.25.4 and OpenSwan, also - but there are no SKB msgs: # lspci 00:00.0 Host bridge: Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub (rev 03) 00:02.0 VGA compatible controller: Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller (rev 03) 00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 02) 00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 02) 00:1c.1 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 2 (rev 02) 00:1c.2 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 3 (rev 02) 00:1c.3 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 4 (rev 02) 00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 02) 00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 02) 00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 02) 00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 02) 00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 02) 00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev e2) 00:1f.0 ISA bridge: Intel Corporation 82801GBM (ICH7-M) LPC Interface Bridge (rev 02) 00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 02) 00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA IDE Controller (rev 02) 00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 02) 01:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller 02:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller 03:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller This bug just hit me today and it seems to have completely locked out all network access to the box requiring a reset button push (headless box). Jul 13 10:44:20 firewall kernel: SKB BUG: Invalid truesize (304) len=136, sizeof(sk_buff)=176 Looking at the logs, I see that it's popped up a few times before this but did not lead to any perceptible detrimental symptoms. We ran 2.6.25.4-10.fc8 for 16 days without any SKB BUG log (and 2.6.24 before that without it also). Then we rebooted into 2.6.25.6-27.fc8 and in 3 days the bug started showing up. Looks like the bug is post-2.6.25.4-10.fc8, unless 16 days wasn't enough time for it to trigger. We are also running PPPoE and ipsec. We are NOT running openswan or raccoon, just a manually setup ipsec.conf and 2.6sec. I have a hunch this bug is related to the PPPoE "waiting for pppX to become free" bug which I've fallen behind on following. A quick check of the logs shows that the "waiting" bug may be fixed, but perhaps this bug has been substituted in its place (same symptoms, loss of all network connectivity). The last kernel log messages before reset button (so some may have been lost) are: Jul 14 11:00:04 firewall kernel: SKB BUG: Invalid truesize (304) len=136, sizeof(sk_buff)=176 Jul 14 11:00:15 firewall kernel:last message repeated 281 times Jul 14 11:00:15 firewall bcrelay[13431]: ignored ENOBUFS from sendto(), temporary shortage of buffer memory Jul 14 11:00:15 firewall bcrelay[13431]:last message repeated 14 times Jul 14 11:00:15 firewall kernel: SKB BUG: Invalid truesize (304) len=136, sizeof(sk_buff)=176 Bug persists also in kernel 2.6.25.14, but no impact in system stability. I can confirm this one. Updating from 2.6.23.12 -> 2.6.26.5 i got this one too. Its a old athlon 800, running ipsec with racoon. I just got a bit of those on kernel 2.6.27.5 on sparc64. SKB BUG: Invalid truesize (288) len=66, sizeof(sk_buff)=224 SKB BUG: Invalid truesize (736) len=515, sizeof(sk_buff)=224 Occurred around the time I restarted the other end of manually configured IPsec tunnel. No other kernel messages showed up. The system looks stable and working. /me too Started after upgrading 2.6.24 -> 2.6.26 Appers on every box running IPSEC (racoon/openswan) and OpenVPN with kernel 2.6.26. Problem escalation since 2.6.28: Dec 29 10:25:33 corebb kernel: ------------[ cut here ]------------ Dec 29 10:25:33 corebb kernel: WARNING: at net/core/skbuff.c:154 sock_rfree+0x64/0x70() Dec 29 10:25:33 corebb kernel: SKB BUG: Invalid truesize (360) len=150, sizeof(sk_buff)=232 Dec 29 10:25:33 corebb kernel: Modules linked in: authenc xfrm4_mode_tunnel ip_gre dummy af_packet cpufreq_ondemand ac battery sg sr_mod cdrom nf_conntrack_ipv6 ip6t_rt ip6t_REJECT ip6table_filter ip6_tables ipv6 xt_tcpudp xt_state ipt_LOG xt_limit ipt_REJECT iptable_filter iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 deflate ctr twofish twofish_common camellia serpent blowfish des_generic aes_x86_64 aes_generic xcbc rmd160 sha256_generic hmac crypto_null af_key it87 w83791d hwmon_vid hwmon eeprom ppp_deflate zlib_deflate zlib_inflate bsd_comp sha1_generic arc4 ecb ppp_mppe ppp_generic slhc powernow_k8 loop usb_storage fan thermal button processor thermal_sys forcedeth serio_raw ohci_hcd ehci_hcd pcspkr evdev i2c_nforce2 i2c_core Dec 29 10:25:33 corebb kernel: Pid: 2741, comm: snort Tainted: G W 2.6.28 #1 Dec 29 10:25:33 corebb kernel: Call Trace: Dec 29 10:25:33 corebb kernel: [<ffffffff8023a6ad>] warn_slowpath+0xcd/0x110 Dec 29 10:25:33 corebb kernel: [<ffffffff80236baa>] dequeue_task_fair+0xca/0xf0 Dec 29 10:25:33 corebb kernel: [<ffffffff80230991>] update_curr+0x51/0xd0 Dec 29 10:25:33 corebb kernel: [<ffffffff80234d8b>] finish_task_switch+0x3b/0xc0 Dec 29 10:25:33 corebb kernel: [<ffffffff8056d504>] thread_return+0x41/0x50d Dec 29 10:25:33 corebb kernel: [<ffffffff80530590>] udp_poll+0x0/0x120 Dec 29 10:25:33 corebb kernel: [<ffffffff803dd914>] __ratelimit+0x4/0xe0 Dec 29 10:25:33 corebb kernel: [<ffffffff803dd9cb>] __ratelimit+0xbb/0xe0 Dec 29 10:25:33 corebb kernel: [<ffffffff804dbbc4>] sock_rfree+0x64/0x70 Dec 29 10:25:33 corebb kernel: [<ffffffff804df59e>] skb_release_head_state+0x4e/0xe0 Dec 29 10:25:33 corebb kernel: [<ffffffff804dfdf9>] skb_release_all+0x9/0x20 Dec 29 10:25:33 corebb kernel: [<ffffffff804df469>] __kfree_skb+0x9/0x80 Dec 29 10:25:33 corebb kernel: [<ffffffff804e24ac>] skb_free_datagram+0xc/0x40 Dec 29 10:25:33 corebb kernel: [<ffffffffa030a993>] packet_recvmsg+0x1b3/0x1e0 [af_packet] Dec 29 10:25:33 corebb kernel: [<ffffffff804d82ea>] sock_recvmsg+0x14a/0x160 Dec 29 10:25:33 corebb kernel: [<ffffffff8024fc10>] autoremove_wake_function+0x0/0x30 Dec 29 10:25:33 corebb kernel: [<ffffffff8023f1b9>] ns_to_timeval+0x9/0x40 Dec 29 10:25:33 corebb kernel: [<ffffffff804d7bb1>] sockfd_lookup_light+0x41/0x80 Dec 29 10:25:33 corebb kernel: [<ffffffff804d99c2>] sys_recvfrom+0xe2/0x190 Dec 29 10:25:33 corebb kernel: [<ffffffff804da2f4>] sock_ioctl+0x54/0x240 Dec 29 10:25:33 corebb kernel: [<ffffffff802b426f>] vfs_ioctl+0x2f/0xa0 Dec 29 10:25:33 corebb kernel: [<ffffffff802b435c>] do_vfs_ioctl+0x7c/0x480 Dec 29 10:25:33 corebb kernel: [<ffffffff8020c20b>] system_call_fastpath+0x16/0x1b Dec 29 10:25:33 corebb kernel: ---[ end trace e92f2c8f5a1520e3 ]--- Dec 29 10:25:42 corebb kernel: hmac crypto_null af_key it87 w83791d hwmon_vid hwmon eeprom ppp_deflate zlib_deflate zlib_inflate bsd_comp sha1_generic arc4 ecb ppp_mppe ppp_generic slhc powernow_k8 loop usb_storage fan thermal button processor thermal_sys forcedeth serio_raw ohci_hcd ehci_hcd pcspkr evdev i2c_nforce2 i2c_coreaarm0c04 Dec 29 10:25:42 corebb kernel: 4 <ffff0093]pce_evs+xb/xe a_akt Dec 29 10:25:42 corebb kernel: 4 <ffff0d2a]sc_evs+x4/x6 Dec 29 10:25:42 corebb kernel: 4 <ffff04c0]atrmv_aefnto+x/x0<>[ffff8320> a_ieicl0001 Dec 29 10:25:42 corebb kernel: 4 <ffff0319]n_otmvl0904 Dec 29 10:25:42 corebb kernel: 4 <ffff0db1]scf_okplgt04/x0<>[ffff849c> y_evrm0e/x9 Dec 29 10:25:42 corebb kernel: 4 <ffff0d24]sc_ot+x4020<>[ffff8246> f_ot+xf0a Dec 29 10:25:42 corebb kernel: 4 <ffff0b3c]d_f_ot+xc040<>[ffff8320> a_ieicl0001 Dec 29 10:25:42 corebb kernel: 4 <ffff002b]sse_alfspt+x601 This happens also on NVidia chipset mainboards. Besides spamming into logfile there seems no impact into system. IPsec is running w/o problems. My recommendation: 1st: If this is a kernel problem, it should be fixed 2nd: If this is not a problem, there should be a method to turn off that syslog spamming ;-) I see this bug on my laptop (working as a router) and every time I see it, the my ethernet iface connected via USB ID 05e9:0008 Kawasaki LSI KL5KUSB101B Ethernet [klsi] (working with kaweth.o) "dies". The other iface (eth0: RealTek RTL-8169) works properly, even when the usb iface dies after "SKB BUG" log entries. I think this is not hardware/related, because i've seen and noticed the same when i was using a pcmcia WiFi card in place of "Kawasaki LSI KL5KUSB101B". I'm NOT USING IPSEC, I use OpenVPN. The only way to fix this for me is to do: `ifconfig eth1 down ; ifconfig eth1 up` and everything works again. I've added this to my crontab, that helps me to keep my router working with this bug: */1 * * * * if ! ping -b -c 1 -w 2 -t 1 -I wlan0 192.168.66.253 1>/dev/null; then ifconfig eth1 down ; sleep 2; ifconfig eth1 up; fi Where 192.168.66.253 is a host that works in my internal network and that should respond. # uname -a Linux aristo2600 2.6.27.6-router #1 Wed Nov 19 23:13:55 CET 2008 i686 Intel(R) Celeron(R) CPU 2.40GHz GenuineIntel GNU/Linux # lspci 00:00.0 Host bridge: Silicon Integrated Systems [SiS] 650/M650 Host (rev 80) 00:01.0 PCI bridge: Silicon Integrated Systems [SiS] Virtual PCI-to-PCI bridge (AGP) 00:02.0 ISA bridge: Silicon Integrated Systems [SiS] SiS962 [MuTIOL Media IO] (rev 14) 00:02.1 SMBus: Silicon Integrated Systems [SiS] SiS961/2 SMBus Controller 00:02.3 FireWire (IEEE 1394): Silicon Integrated Systems [SiS] FireWire Controller 00:02.5 IDE interface: Silicon Integrated Systems [SiS] 5513 [IDE] 00:02.6 Modem: Silicon Integrated Systems [SiS] AC'97 Modem Controller (rev a0) 00:02.7 Multimedia audio controller: Silicon Integrated Systems [SiS] AC'97 Sound Controller (rev a0) 00:03.0 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f) 00:03.1 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f) 00:03.2 USB Controller: Silicon Integrated Systems [SiS] USB 1.1 Controller (rev 0f) 00:03.3 USB Controller: Silicon Integrated Systems [SiS] USB 2.0 Controller 00:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 00:0c.0 CardBus bridge: ENE Technology Inc CB1410 Cardbus Controller (rev 01) 01:00.0 VGA compatible controller: Silicon Integrated Systems [SiS] 65x/M650/740 PCI/AGP VGA Display Adapter # lsusb Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 008: ID 05e9:0008 Kawasaki LSI KL5KUSB101B Ethernet [klsi] Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub # lsmod Module Size Used by sch_sfq 6144 0 cls_u32 6656 0 sch_htb 15488 0 iptable_mangle 2688 0 tun 10116 2 xt_tcpudp 2816 14 xt_limit 2048 2 xt_state 1920 9 ipt_REJECT 2944 1 ipt_LOG 5504 3 iptable_nat 5128 1 nf_nat 15120 1 iptable_nat nf_conntrack_ipv4 12812 12 iptable_nat,nf_nat nf_conntrack 49696 4 xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4 iptable_filter 2560 1 ip_tables 10256 3 iptable_mangle,iptable_nat,iptable_filter x_tables 12164 7 xt_tcpudp,xt_limit,xt_state,ipt_REJECT,ipt_LOG,iptable_nat,ip_tables snd_pcsp 8836 0 snd_pcm 64392 1 snd_pcsp rtc_cmos 9376 0 rtc_core 14872 1 rtc_cmos snd_timer 18948 1 snd_pcm rtc_lib 2432 1 rtc_core floppy 52356 0 snd 45988 3 snd_pcsp,snd_pcm,snd_timer soundcore 5472 1 snd snd_page_alloc 7304 1 snd_pcm psmouse 35984 0 serio_raw 5124 0 8139too 22272 0 yenta_socket 22796 0 mii 4480 1 8139too rsrc_nonstatic 10368 1 yenta_socket pcmcia_core 30992 2 yenta_socket,rsrc_nonstatic 8250_pnp 13952 0 8250_pci 22784 0 kaweth 12288 0 nsc_ircc 15632 0 irda 100536 1 nsc_ircc crc_ccitt 1792 1 irda ide_cd_mod 30596 0 8250 21620 2 8250_pnp,8250_pci serial_core 18304 1 8250 container 3328 0 firewire_ohci 20740 0 firewire_core 35608 1 firewire_ohci ohci1394 28208 0 ieee1394 77252 1 ohci1394 battery 9988 0 ac 3972 0 thermal 15004 0 button 5904 0 processor 27952 1 thermal thermal_sys 11040 2 thermal,processor hwmon 2196 1 thermal_sys i2c_sis96x 4228 0 shpchp 28692 0 pci_hotplug 12296 1 shpchp (In reply to comment #11) > I see this bug on my laptop (working as a router) and every time I see it, > the > my ethernet iface connected via USB > > ID 05e9:0008 Kawasaki LSI KL5KUSB101B Ethernet [klsi] (working with kaweth.o) > > "dies". The other iface (eth0: RealTek RTL-8169) works properly, even when > the > usb iface dies after "SKB BUG" log entries. I think this is not > hardware/related, because i've seen and noticed the same when i was using a > pcmcia WiFi card in place of "Kawasaki LSI KL5KUSB101B". > > I'm NOT USING IPSEC, I use OpenVPN. So this is a different bug, probably in kaweth. Please consider creating separate bugreport. Bug still exists on Kernel 2.6.28.4, can be verified on different HW platforms, at least on Intel Core2 and AMD FX62. Following output from AMD/NVidia mainboard. # uname -a Linux corebb 2.6.28.4 #1 SMP PREEMPT Sun Feb 8 11:12:25 CET 2009 x86_64 GNU/Linux # lspci 00:00.0 RAM memory: nVidia Corporation C51 Host Bridge (rev a2) 00:00.1 RAM memory: nVidia Corporation C51 Memory Controller 0 (rev a2) 00:00.2 RAM memory: nVidia Corporation C51 Memory Controller 1 (rev a2) 00:00.3 RAM memory: nVidia Corporation C51 Memory Controller 5 (rev a2) 00:00.4 RAM memory: nVidia Corporation C51 Memory Controller 4 (rev a2) 00:00.5 RAM memory: nVidia Corporation C51 Host Bridge (rev a2) 00:00.6 RAM memory: nVidia Corporation C51 Memory Controller 3 (rev a2) 00:00.7 RAM memory: nVidia Corporation C51 Memory Controller 2 (rev a2) 00:04.0 PCI bridge: nVidia Corporation C51 PCI Express Bridge (rev a1) 00:08.0 RAM memory: nVidia Corporation MCP55 Memory Controller (rev a1) 00:09.0 ISA bridge: nVidia Corporation MCP55 LPC Bridge (rev a2) 00:09.1 SMBus: nVidia Corporation MCP55 SMBus (rev a2) 00:0a.0 USB Controller: nVidia Corporation MCP55 USB Controller (rev a1) 00:0a.1 USB Controller: nVidia Corporation MCP55 USB Controller (rev a2) 00:0c.0 IDE interface: nVidia Corporation MCP55 IDE (rev a1) 00:0d.0 IDE interface: nVidia Corporation MCP55 SATA Controller (rev a2) 00:0d.1 IDE interface: nVidia Corporation MCP55 SATA Controller (rev a2) 00:0d.2 IDE interface: nVidia Corporation MCP55 SATA Controller (rev a2) 00:0e.0 PCI bridge: nVidia Corporation MCP55 PCI bridge (rev a2) 00:10.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2) 00:11.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2) 00:12.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev a2) 00:16.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev a2) 00:17.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev a2) 00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration 00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map 00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller 00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control 01:00.0 VGA compatible controller: nVidia Corporation GeForce 7300 GS (rev a1) 02:0b.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link) 04:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01) corebb:/usr/src# lsusb Bus 002 Device 002: ID 14dd:0002 Bus 002 Device 001: ID 1d6b:0002 Bus 001 Device 001: ID 1d6b:0001 # lsmod Module Size Used by authenc 8448 12 xfrm4_mode_tunnel 4928 24 xfrm_user 26560 2 xfrm4_tunnel 4928 0 af_key 35224 0 ip_gre 18372 0 dummy 5120 0 af_packet 22336 2 cpufreq_ondemand 9872 0 ac 7240 0 battery 14856 0 nf_conntrack_ipv6 17480 1 ip6t_rt 5056 3 ip6t_REJECT 6528 4 ip6table_filter 5312 1 ip6_tables 22416 2 ip6t_rt,ip6table_filter ipv6 327024 46 nf_conntrack_ipv6,ip6t_REJECT xt_tcpudp 5696 53 xt_state 4544 3 ipt_LOG 8516 5 xt_limit 4996 4 ipt_REJECT 5696 4 iptable_filter 5440 1 iptable_nat 8516 1 nf_nat 23828 1 iptable_nat nf_conntrack_ipv4 18072 5 iptable_nat,nf_nat nf_conntrack 76624 5 nf_conntrack_ipv6,xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4 nf_defrag_ipv4 4480 1 nf_conntrack_ipv4 ip_tables 20816 2 iptable_filter,iptable_nat x_tables 25480 10 ip6t_rt,ip6t_REJECT,ip6_tables,xt_tcpudp,xt_state,ipt_LOG,xt_limit,ipt_REJECT,iptable_nat,ip_tables tunnel4 5968 1 xfrm4_tunnel ipcomp 5632 0 xfrm_ipcomp 8396 1 ipcomp esp4 9216 12 ah4 7744 0 deflate 5440 0 ctr 6976 0 twofish 8576 0 twofish_common 16704 1 twofish camellia 21312 0 serpent 21376 0 blowfish 10816 0 des_generic 19392 0 aes_x86_64 10368 12 aes_generic 30568 1 aes_x86_64 xcbc 7560 0 rmd160 10048 0 sha256_generic 11712 0 hmac 6784 12 crypto_null 5696 0 sg 34656 0 sr_mod 18948 0 cdrom 40104 1 sr_mod it87 28888 0 w83791d 27228 0 hwmon_vid 5888 2 it87,w83791d hwmon 5256 2 it87,w83791d eeprom 8720 0 ppp_deflate 7424 0 zlib_deflate 23448 2 deflate,ppp_deflate zlib_inflate 18432 2 deflate,ppp_deflate bsd_comp 7872 0 sha1_generic 4928 12 arc4 4160 0 ecb 5312 0 ppp_mppe 9352 0 ppp_generic 28584 3 ppp_deflate,bsd_comp,ppp_mppe slhc 8896 1 ppp_generic powernow_k8 16068 1 loop 19020 0 usb_storage 97472 0 fan 7496 0 thermal 20896 0 i2c_nforce2 9664 0 button 9632 0 processor 48184 2 powernow_k8,thermal thermal_sys 14464 3 fan,thermal,processor serio_raw 8260 0 forcedeth 60944 0 ehci_hcd 39052 0 ohci_hcd 27100 0 i2c_core 29664 3 w83791d,eeprom,i2c_nforce2 pcspkr 5120 0 evdev 13856 0 [ 309.701229] ------------[ cut here ]------------ [ 309.701271] WARNING: at net/core/skbuff.c:154 sock_rfree+0x64/0x70() [ 309.701312] SKB BUG: Invalid truesize (360) len=158, sizeof(sk_buff)=232 [ 309.701352] Modules linked in: authenc xfrm4_mode_tunnel xfrm_user xfrm4_tunnel af_key ip_gre dummy af_packet cpufreq_ondemand ac battery nf_conntrack_ipv6 ip6t_rt ip6t_REJECT ip6table_filter ip6_tables ipv6 xt_tcpudp xt_state ipt_LOG xt_limit ipt_REJECT iptable_filter iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables tunnel4 ipcomp xfrm_ipcomp esp4 ah4 deflate ctr twofish twofish_common camellia serpent blowfish des_generic aes_x86_64 aes_generic xcbc rmd160 sha256_generic hmac crypto_null sg sr_mod cdrom it87 w83791d hwmon_vid hwmon eeprom ppp_deflate zlib_deflate zlib_inflate bsd_comp sha1_generic arc4 ecb ppp_mppe ppp_generic slhc powernow_k8 loop usb_storage fan thermal i2c_nforce2 button processor thermal_sys serio_raw forcedeth ehci_hcd ohci_hcd i2c_core pcspkr evdev [last unloaded: xfrm_user] [ 309.703604] Pid: 2697, comm: snort Tainted: G W 2.6.28.4 #1 [ 309.703644] Call Trace: [ 309.703684] [<ffffffff8023a73d>] warn_slowpath+0xcd/0x110 [ 309.703724] [<ffffffff80236c3a>] dequeue_task_fair+0xca/0xf0 [ 309.703764] [<ffffffff80230f7d>] __dequeue_entity+0x3d/0x50 [ 309.703804] [<ffffffff80230fb5>] set_next_entity+0x25/0x50 [ 309.703844] [<ffffffff8056d704>] thread_return+0x41/0x50d [ 309.703885] [<ffffffff803a23b0>] cap_socket_recvmsg+0x0/0x10 [ 309.703925] [<ffffffff8056de85>] schedule_timeout+0x95/0xd0 [ 309.703966] [<ffffffff8024fdf3>] prepare_to_wait_exclusive+0x23/0x70 [ 309.704012] [<ffffffff804e24cb>] __skb_recv_datagram+0x1eb/0x280 [ 309.704053] [<ffffffff804e17b8>] memcpy_toiovec+0x8/0x70 [ 309.704093] [<ffffffff803dda4b>] __ratelimit+0xbb/0xe0 [ 309.704140] [<ffffffff804dbdb4>] sock_rfree+0x64/0x70 [ 309.704185] [<ffffffff804df78e>] skb_release_head_state+0x4e/0xe0 [ 309.704247] [<ffffffff804dffe9>] skb_release_all+0x9/0x20 [ 309.704304] [<ffffffff804df659>] __kfree_skb+0x9/0x80 [ 309.704345] [<ffffffff804e269c>] skb_free_datagram+0xc/0x40 [ 309.704388] [<ffffffffa030a993>] packet_recvmsg+0x1b3/0x1e0 [af_packet] [ 309.704429] [<ffffffff804d84da>] sock_recvmsg+0x14a/0x160 [ 309.704476] [<ffffffff8024fce0>] autoremove_wake_function+0x0/0x30 [ 309.704534] [<ffffffff8023f259>] ns_to_timeval+0x9/0x40 [ 309.704587] [<ffffffff804d7da1>] sockfd_lookup_light+0x41/0x80 [ 309.704633] [<ffffffff804d9bb2>] sys_recvfrom+0xe2/0x190 [ 309.704672] [<ffffffff804da4e4>] sock_ioctl+0x54/0x240 [ 309.704711] [<ffffffff802b419f>] vfs_ioctl+0x2f/0xa0 [ 309.704750] [<ffffffff802b428c>] do_vfs_ioctl+0x7c/0x480 [ 309.704801] [<ffffffff8020c20b>] system_call_fastpath+0x16/0x1b [ 309.704860] ---[ end trace 358b71b4626cc696 ]--- Is there any way to turn off that logs if problem does not impact system performance / stability ? Turning off IPsec is not an option. What the status of this bug? I'm hitting it recently. Linux anthrax 2.6.27 #1 Fri Oct 10 17:16:44 CEST 2008 i586 Pentium MMX GenuineIntel GNU/Linux It's pentium166mmx with 2x e100 and 1xe1000, working mainly as router (IPv4 ethernet/IPv6 aiccu tunel froms Sixxs). 160 MB of RAM, irssi, mldonkey, samba, ftp, maradns, apache, postgresql. IPSec is selected in .config, but not used in anyway. The same for traffic control. Netfilter has about 2 dozen of rules. This message appears often in logs, but no connection is broken and machine still works. % dmesg -s 10000 | sort -u SKB BUG: Invalid truesize (272) len=98, sizeof(sk_buff)=176 SKB BUG: Invalid truesize (304) len=134, sizeof(sk_buff)=176 SKB BUG: Invalid truesize (400) len=230, sizeof(sk_buff)=176 % dmesg -s 10000 | grep -c "SKB BUG" 156 I have the same problem with 2.6.28.7. [828090.107912] ------------[ cut here ]------------ [828090.161915] WARNING: at net/core/skbuff.c:150 sock_rfree+0x22/0x51() [828090.235950] SKB BUG: Invalid truesize (360) len=134, sizeof(sk_buff)=232 [828090.313892] Modules linked in: tg3 esp6 xfrm6_mode_tunnel authenc esp4 xfrm4_mode_tunnel tun kvm_intel kvm xt_NOTRACK ip6table_raw ip6t_LOG nf_conntrack_ipv6 ip6table_filter ip6_tables xt_physdev ipt_LOG xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_hashlimit iptable_filter ip_tables x_tables bridge stp llc deflate zlib_deflate zlib_inflate ctr twofish twofish_common camellia serpent blowfish des_generic cbc aes_x86_64 aes_generic xcbc sha256_generic sha1_generic crypto_null af_key dm_crypt ipv6 coretemp loop ipmi_si ipmi_msghandler hpilo hpwdt pcspkr shpchp pci_hotplug container button psmouse serio_raw evdev ext3 jbd dm_mirror dm_region_hash dm_log dm_snapshot dm_mod raid1 md_mod sg sd_mod sr_mod cdrom usbhid hid ata_piix ata_generic libata scsi_mod ide_pci_generic ide_core ehci_hcd libphy uhci_hcd thermal processor fan thermal_sys [last unloaded: tg3] [828091.208842] Pid: 25130, comm: tcpdump Tainted: G W 2.6.28.7 #1 [828091.284891] Call Trace: [828091.313855] [<ffffffff802398aa>] warn_slowpath+0xb4/0xda [828091.376854] [<ffffffff80235a9a>] finish_task_switch+0x2b/0xc4 [828091.444884] [<ffffffff8043885a>] thread_return+0x3d/0xc5 [828091.507894] [<ffffffff80377169>] n_tty_receive_buf+0xd1e/0xd6b [828091.576873] [<ffffffff80243f63>] __dequeue_signal+0xfb/0x124 [828091.643860] [<ffffffff802454c3>] dequeue_signal+0x9c/0x113 [828091.708918] [<ffffffff804389e0>] schedule_timeout+0x1e/0xad [828091.775878] [<ffffffff80439e36>] _spin_lock_irq+0xd/0xf [828091.837859] [<ffffffff8020b8d6>] do_notify_resume+0x718/0x837 [828091.905859] [<ffffffff80379445>] pty_write_room+0x0/0x1f [828091.968877] [<ffffffff80439e36>] _spin_lock_irq+0xd/0xf [828092.030908] [<ffffffff80438125>] wait_for_common+0x129/0x14d [828092.097902] [<ffffffff80221be9>] default_spin_lock_flags+0x5/0x8 [828092.168919] [<ffffffff80439e21>] _spin_lock_irqsave+0x24/0x2c [828092.236923] [<ffffffff80335e51>] __ratelimit+0xb5/0xc0 [828092.297980] [<ffffffff803c151d>] sock_rfree+0x22/0x51 [828092.357907] [<ffffffff80439e21>] _spin_lock_irqsave+0x24/0x2c [828092.425904] [<ffffffff803c5056>] skb_release_head_state+0x60/0xbb [828092.497888] [<ffffffff803c476c>] __kfree_skb+0x9/0x77 [828092.557868] [<ffffffff803c4809>] skb_queue_purge+0xb/0x1d [828092.621887] [<ffffffff80428d5f>] packet_release+0x16f/0x18f [828092.687841] [<ffffffff8043885a>] thread_return+0x3d/0xc5 [828092.750858] [<ffffffff803bf06a>] sock_release+0x19/0x72 [828092.813834] [<ffffffff803bf0e5>] sock_close+0x22/0x27 [828092.873839] [<ffffffff8029f82d>] __fput+0xc6/0x18a [828092.930840] [<ffffffff8029d129>] filp_close+0x50/0x57 [828092.990823] [<ffffffff8029d1cb>] sys_close+0x9b/0xdb [828093.049845] [<ffffffff8020be4a>] system_call_fastpath+0x16/0x1b [828093.119856] ---[ end trace 444b547394c96982 ]--- I think it's related to IPsec here as well, I only hit this BUG when I run tcpdump on the external interface and it sees an ESP packet. Should be fixed with the next -stable releases (2.6.27.20 and 26.28.8): http://git.kernel.org/?p=linux/kernel/git/stable/stable-queue.git;a=blob;f=review-2.6.27/net-kill-skb_truesize_check-it-only-catches-false-positives.patch;h=14296ae60663c4ff4bce76a3145918a7cd7d7a87;hb=HEAD http://git.kernel.org/?p=linux/kernel/git/stable/stable-queue.git;a=blob;f=review-2.6.28/net-kill-skb_truesize_check-it-only-catches-false-positives.patch;h=5f205fc7e6736adca699635e695a61798624b172;hb=HEAD |