Bug 8797

Summary: WARNING from skb_gso_segment
Product: Networking Reporter: Howard Kash (hmkash)
Component: IPV4Assignee: Herbert Xu (herbert)
Status: RESOLVED CODE_FIX    
Severity: normal CC: benny+bugzilla, herbert
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 2.6.22.1 Subsystem:
Regression: --- Bisected commit-id:
Attachments: [IPVS]: Do not clobber ip_summed on xmit
[NET]: Share correct feature code between bridging and bonding
[NET]: Share correct feature code between bridging and bonding

Description Howard Kash 2007-07-23 09:41:40 UTC 
Most recent kernel where this bug did not occur: 2.6.20.4

Distribution: kernel.org

Hardware Environment: x86, dual-core Xeon, tg3, e1000

Software Environment: ip_vs, bonding

Problem Description: After upgrading kernels from 2.6.20.4 to 2.6.22.1, I started getting tons (27,000+ a day) of the warning messages below resulting from this change:

http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.22.y.git;a=commitdiff;h=f9d106a6d53b57b78eae5544f9582c643343a764

This is on a system that uses ip_vs to load balance DNS traffic. Other than these warnings, everything appears to be normal with the load balanced DNS traffic.

Jul 17 09:29:40 hlb1 kernel: WARNING: at net/core/dev.c:1286 skb_gso_segment()
Jul 17 09:29:40 hlb1 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
Jul 17 09:29:40 hlb1 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
Jul 17 09:29:40 hlb1 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
Jul 17 09:29:40 hlb1 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
Jul 17 09:29:40 hlb1 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 17 09:29:40 hlb1 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 17 09:29:40 hlb1 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
Jul 17 09:29:40 hlb1 kernel:  [<e0a2550c>] do_ip_vs_get_ctl+0x6f5/0x704 [ip_vs]
Jul 17 09:29:40 hlb1 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 17 09:29:40 hlb1 kernel:  [<e0a2687f>] ip_vs_dr_xmit+0x345/0x37b [ip_vs]
Jul 17 09:29:40 hlb1 kernel:  [<c02a336a>] _spin_lock_bh+0x8/0x10
Jul 17 09:29:40 hlb1 kernel:  [<c0276547>] tcp_cwnd_restart+0x17/0xad
Jul 17 09:29:40 hlb1 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
Jul 17 09:29:40 hlb1 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
Jul 17 09:29:40 hlb1 kernel:  [<c0277f91>] tcp_write_xmit+0x1cc/0x22e
Jul 17 09:29:40 hlb1 kernel:  [<c0278004>] __tcp_push_pending_frames+0x11/0x60
Jul 17 09:29:40 hlb1 kernel:  [<c026e430>] tcp_sendmsg+0x9df/0xa95
Jul 17 09:29:40 hlb1 kernel:  [<c0287133>] inet_sendmsg+0x39/0x43
Jul 17 09:29:40 hlb1 kernel:  [<c0243be2>] do_sock_write+0xab/0xb2
Jul 17 09:29:40 hlb1 kernel:  [<c0243c39>] sock_aio_write+0x50/0x5c
Jul 17 09:29:40 hlb1 kernel:  [<c015c008>] do_sync_write+0xbf/0xfc
Jul 17 09:29:40 hlb1 kernel:  [<c02a2b70>] mutex_lock+0x13/0x22
Jul 17 09:29:40 hlb1 kernel:  [<c0129232>] autoremove_wake_function+0x0/0x33
Jul 17 09:29:40 hlb1 kernel:  [<c02472ad>] sock_common_getsockopt+0x1c/0x21
Jul 17 09:29:40 hlb1 kernel:  [<c0244c49>] sys_getsockopt+0x7d/0x9c
Jul 17 09:29:40 hlb1 kernel:  [<c015c10e>] vfs_write+0xc9/0x133
Jul 17 09:29:40 hlb1 kernel:  [<c015c220>] sys_write+0x41/0x67
Jul 17 09:29:40 hlb1 kernel:  [<c0103002>] sysenter_past_esp+0x5f/0x85
Jul 17 09:29:40 hlb1 kernel:  =======================


Steps to reproduce: Unsure
Comment 1 Anonymous Emailer 2007-07-23 12:55:09 UTC 
Reply-To: akpm@linux-foundation.org

On Mon, 23 Jul 2007 09:36:08 -0700 (PDT)
bugme-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=8797
> 
>            Summary: WARNING from skb_gso_segment
>            Product: Networking
>            Version: 2.5
>      KernelVersion: 2.6.22.1
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: normal
>           Priority: P1
>          Component: IPV4
>         AssignedTo: shemminger@osdl.org
>         ReportedBy: hmkash@arl.army.mil
>                 CC: herbert@gondor.apana.org.au
> 
> 
> Most recent kernel where this bug did not occur: 2.6.20.4
> 
> Distribution: kernel.org
> 
> Hardware Environment: x86, dual-core Xeon, tg3, e1000
> 
> Software Environment: ip_vs, bonding
> 
> Problem Description: After upgrading kernels from 2.6.20.4 to 2.6.22.1, I
> started getting tons (27,000+ a day) of the warning messages below resulting
> from this change:
> 
>
> http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.22.y.git;a=commitdiff;h=f9d106a6d53b57b78eae5544f9582c643343a764
> 
> This is on a system that uses ip_vs to load balance DNS traffic. Other than
> these warnings, everything appears to be normal with the load balanced DNS
> traffic.
> 
> Jul 17 09:29:40 hlb1 kernel: WARNING: at net/core/dev.c:1286
> skb_gso_segment()
> Jul 17 09:29:40 hlb1 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
> Jul 17 09:29:40 hlb1 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
> Jul 17 09:29:40 hlb1 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
> Jul 17 09:29:40 hlb1 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
> Jul 17 09:29:40 hlb1 kernel:  [<c0267828>] ip_output+0x274/0x2ad
> Jul 17 09:29:40 hlb1 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
> Jul 17 09:29:40 hlb1 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
> Jul 17 09:29:40 hlb1 kernel:  [<e0a2550c>] do_ip_vs_get_ctl+0x6f5/0x704
> [ip_vs]
> Jul 17 09:29:40 hlb1 kernel:  [<c0267828>] ip_output+0x274/0x2ad
> Jul 17 09:29:40 hlb1 kernel:  [<e0a2687f>] ip_vs_dr_xmit+0x345/0x37b [ip_vs]
> Jul 17 09:29:40 hlb1 kernel:  [<c02a336a>] _spin_lock_bh+0x8/0x10
> Jul 17 09:29:40 hlb1 kernel:  [<c0276547>] tcp_cwnd_restart+0x17/0xad
> Jul 17 09:29:40 hlb1 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
> Jul 17 09:29:40 hlb1 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
> Jul 17 09:29:40 hlb1 kernel:  [<c0277f91>] tcp_write_xmit+0x1cc/0x22e
> Jul 17 09:29:40 hlb1 kernel:  [<c0278004>]
> __tcp_push_pending_frames+0x11/0x60
> Jul 17 09:29:40 hlb1 kernel:  [<c026e430>] tcp_sendmsg+0x9df/0xa95
> Jul 17 09:29:40 hlb1 kernel:  [<c0287133>] inet_sendmsg+0x39/0x43
> Jul 17 09:29:40 hlb1 kernel:  [<c0243be2>] do_sock_write+0xab/0xb2
> Jul 17 09:29:40 hlb1 kernel:  [<c0243c39>] sock_aio_write+0x50/0x5c
> Jul 17 09:29:40 hlb1 kernel:  [<c015c008>] do_sync_write+0xbf/0xfc
> Jul 17 09:29:40 hlb1 kernel:  [<c02a2b70>] mutex_lock+0x13/0x22
> Jul 17 09:29:40 hlb1 kernel:  [<c0129232>] autoremove_wake_function+0x0/0x33
> Jul 17 09:29:40 hlb1 kernel:  [<c02472ad>] sock_common_getsockopt+0x1c/0x21
> Jul 17 09:29:40 hlb1 kernel:  [<c0244c49>] sys_getsockopt+0x7d/0x9c
> Jul 17 09:29:40 hlb1 kernel:  [<c015c10e>] vfs_write+0xc9/0x133
> Jul 17 09:29:40 hlb1 kernel:  [<c015c220>] sys_write+0x41/0x67
> Jul 17 09:29:40 hlb1 kernel:  [<c0103002>] sysenter_past_esp+0x5f/0x85
> Jul 17 09:29:40 hlb1 kernel:  =======================
> 
> 
> Steps to reproduce: Unsure
> 
> 
> -- 
> Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug, or are watching someone who is.
Comment 2 Herbert Xu 2007-07-25 22:54:22 UTC 
Created attachment 12148 [details]
[IPVS]: Do not clobber ip_summed on xmit

This patch replaces the unconditinoal setting of ip_summed to CHECKSUM_NONE on xmit with skb_forward_csum.  The latter will preserve CHECKSUM_PARTIAL instead of clobbering it.

This bug has always existed.  It was previously masked by the fact that netfilter also clobbered ip_summed.  Now that netfilter doesn't do that anymore this has risen to the surface.
Comment 3 Herbert Xu 2007-07-25 23:43:50 UTC 
What NAT rules if any do you have in place?
Comment 4 Howard Kash 2007-07-26 06:15:44 UTC 
Classification:  UNCLASSIFIED 
Caveats: NONE


I do not have any NAT rules (or any other netfilter rules for that
matter).

I applied the patch, but I'm still getting warnings but with various
different tracebacks (samples below).  Maybe there's some other
lingering checksum bugs in ip_vs?


Jul 26 08:56:23 hlb2 kernel: WARNING: at net/core/dev.c:1286
skb_gso_segment()
Jul 26 08:56:23 hlb2 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
Jul 26 08:56:23 hlb2 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
Jul 26 08:56:23 hlb2 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
Jul 26 08:56:23 hlb2 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
Jul 26 08:56:23 hlb2 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 08:56:23 hlb2 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 26 08:56:23 hlb2 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
Jul 26 08:56:23 hlb2 kernel:  [<c024d2a6>] dev_hard_start_xmit+0x5e/0xb1
Jul 26 08:56:23 hlb2 kernel:  [<c024d4ee>] dev_queue_xmit+0x1f5/0x1fd
Jul 26 08:56:23 hlb2 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 08:56:23 hlb2 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 26 08:56:23 hlb2 kernel:  [<e0b7b8b7>] ip_vs_dr_xmit+0x353/0x389
[ip_vs]
Jul 26 08:56:23 hlb2 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
Jul 26 08:56:23 hlb2 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
Jul 26 08:56:23 hlb2 kernel:  [<c0277f91>] tcp_write_xmit+0x1cc/0x22e
Jul 26 08:56:23 hlb2 kernel:  [<c0278004>]
__tcp_push_pending_frames+0x11/0x60
Jul 26 08:56:23 hlb2 kernel:  [<c026e430>] tcp_sendmsg+0x9df/0xa95
Jul 26 08:56:23 hlb2 kernel:  [<c02a3358>] _spin_lock_irqsave+0x9/0xd
Jul 26 08:56:23 hlb2 kernel:  [<c0120440>] __mod_timer+0x90/0x9a
Jul 26 08:56:23 hlb2 kernel:  [<e0b7840e>] ip_vs_in+0x1fa/0x204 [ip_vs]
Jul 26 08:56:23 hlb2 kernel:  [<c0287133>] inet_sendmsg+0x39/0x43
Jul 26 08:56:23 hlb2 kernel:  [<c0243be2>] do_sock_write+0xab/0xb2
Jul 26 08:56:23 hlb2 kernel:  [<c0243c39>] sock_aio_write+0x50/0x5c
Jul 26 08:56:23 hlb2 kernel:  [<c015c008>] do_sync_write+0xbf/0xfc
Jul 26 08:56:23 hlb2 kernel:  [<e08d8de9>] tg3_rx+0x340/0x36a [tg3]
Jul 26 08:56:23 hlb2 kernel:  [<c0129232>]
autoremove_wake_function+0x0/0x33
Jul 26 08:56:23 hlb2 kernel:  [<e08d553e>] tg3_restart_ints+0x11/0x5a
[tg3]
Jul 26 08:56:23 hlb2 kernel:  [<e08d8f86>] tg3_poll+0x173/0x180 [tg3]
Jul 26 08:56:23 hlb2 kernel:  [<c024dac0>] net_rx_action+0x8f/0x13b
Jul 26 08:56:23 hlb2 kernel:  [<c015c10e>] vfs_write+0xc9/0x133
Jul 26 08:56:23 hlb2 kernel:  [<c015c220>] sys_write+0x41/0x67
Jul 26 08:56:23 hlb2 kernel:  [<c0103002>] sysenter_past_esp+0x5f/0x85
Jul 26 08:56:23 hlb2 kernel:  =======================



Jul 26 08:56:25 hlb2 kernel: WARNING: at net/core/dev.c:1286
skb_gso_segment()
Jul 26 08:56:25 hlb2 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
Jul 26 08:56:25 hlb2 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
Jul 26 08:56:25 hlb2 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
Jul 26 08:56:25 hlb2 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
Jul 26 08:56:25 hlb2 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 08:56:25 hlb2 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 26 08:56:25 hlb2 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
Jul 26 08:56:25 hlb2 kernel:  [<e0938cc7>] e1000_xmit_frame+0x386/0x390
[e1000]
Jul 26 08:56:25 hlb2 kernel:  [<c024d2a6>] dev_hard_start_xmit+0x5e/0xb1
Jul 26 08:56:25 hlb2 kernel:  [<c0258fd4>] __qdisc_run+0xe8/0x19d
Jul 26 08:56:25 hlb2 kernel:  [<c024d4ee>] dev_queue_xmit+0x1f5/0x1fd
Jul 26 08:56:25 hlb2 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
Jul 26 08:56:25 hlb2 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
Jul 26 08:56:25 hlb2 kernel:  [<c0277f91>] tcp_write_xmit+0x1cc/0x22e
Jul 26 08:56:25 hlb2 kernel:  [<c0278004>]
__tcp_push_pending_frames+0x11/0x60
Jul 26 08:56:25 hlb2 kernel:  [<c02754d8>]
tcp_rcv_established+0xf0/0x59e
Jul 26 08:56:25 hlb2 kernel:  [<e0b782bb>] ip_vs_in+0xa7/0x204 [ip_vs]
Jul 26 08:56:25 hlb2 kernel:  [<c027c1f8>] tcp_v4_do_rcv+0x22/0xbe
Jul 26 08:56:25 hlb2 kernel:  [<c027c71e>] tcp_v4_rcv+0x48a/0x74c
Jul 26 08:56:25 hlb2 kernel:  [<c02642ed>] ip_local_deliver+0x139/0x1e2
Jul 26 08:56:25 hlb2 kernel:  [<c0264396>]
ip_local_deliver_finish+0x0/0x15e
Jul 26 08:56:25 hlb2 kernel:  [<c0264979>] ip_rcv+0x485/0x4bd
Jul 26 08:56:25 hlb2 kernel:  [<c024d926>] netif_receive_skb+0x218/0x239
Jul 26 08:56:25 hlb2 kernel:  [<e0939fdb>]
e1000_clean_rx_irq+0x3b7/0x47a [e1000]
Jul 26 08:56:25 hlb2 kernel:  [<e093988a>] e1000_clean+0x74/0xf6 [e1000]
Jul 26 08:56:25 hlb2 kernel:  [<c024dac0>] net_rx_action+0x8f/0x13b
Jul 26 08:56:25 hlb2 kernel:  [<c011d68c>] __do_softirq+0x63/0xc1
Jul 26 08:56:25 hlb2 kernel:  [<c011d71b>] do_softirq+0x31/0x35
Jul 26 08:56:25 hlb2 kernel:  [<c0105242>] do_IRQ+0x80/0x93
Jul 26 08:56:25 hlb2 kernel:  [<c0103a13>] common_interrupt+0x23/0x28
Jul 26 08:56:25 hlb2 kernel:  [<c014af35>] unmap_page_range+0x57/0x100
Jul 26 08:56:25 hlb2 kernel:  [<c014b0e1>] unmap_vmas+0x103/0x1cf
Jul 26 08:56:25 hlb2 kernel:  [<c014f55c>] exit_mmap+0x6c/0xe2
Jul 26 08:56:25 hlb2 kernel:  [<c0117134>] mmput+0x2f/0x87
Jul 26 08:56:25 hlb2 kernel:  [<c011b90e>] do_exit+0x1d0/0x39a
Jul 26 08:56:25 hlb2 kernel:  [<c011bb67>] sys_exit_group+0x0/0xd
Jul 26 08:56:25 hlb2 kernel:  [<c0103002>] sysenter_past_esp+0x5f/0x85
Jul 26 08:56:25 hlb2 kernel:  [<c02a0000>] tpacket_rcv+0x2d8/0x403
Jul 26 08:56:25 hlb2 kernel:  =======================



Jul 26 08:56:26 hlb2 kernel: WARNING: at net/core/dev.c:1286
skb_gso_segment()
Jul 26 08:56:26 hlb2 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
Jul 26 08:56:26 hlb2 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
Jul 26 08:56:26 hlb2 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
Jul 26 08:56:26 hlb2 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
Jul 26 08:56:26 hlb2 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 08:56:26 hlb2 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 26 08:56:26 hlb2 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
Jul 26 08:56:26 hlb2 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 08:56:26 hlb2 kernel:  [<e0b7b8b7>] ip_vs_dr_xmit+0x353/0x389
[ip_vs]
Jul 26 08:56:26 hlb2 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 08:56:26 hlb2 kernel:  [<e0b7b8b7>] ip_vs_dr_xmit+0x353/0x389
[ip_vs]
Jul 26 08:56:26 hlb2 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
Jul 26 08:56:26 hlb2 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
Jul 26 08:56:26 hlb2 kernel:  [<c0277f91>] tcp_write_xmit+0x1cc/0x22e
Jul 26 08:56:26 hlb2 kernel:  [<c0278004>]
__tcp_push_pending_frames+0x11/0x60
Jul 26 08:56:26 hlb2 kernel:  [<c02754d8>]
tcp_rcv_established+0xf0/0x59e
Jul 26 08:56:26 hlb2 kernel:  [<e0b782bb>] ip_vs_in+0xa7/0x204 [ip_vs]
Jul 26 08:56:26 hlb2 kernel:  [<c027c1f8>] tcp_v4_do_rcv+0x22/0xbe
Jul 26 08:56:26 hlb2 kernel:  [<c027c71e>] tcp_v4_rcv+0x48a/0x74c
Jul 26 08:56:26 hlb2 kernel:  [<c02642ed>] ip_local_deliver+0x139/0x1e2
Jul 26 08:56:26 hlb2 kernel:  [<c0264396>]
ip_local_deliver_finish+0x0/0x15e
Jul 26 08:56:26 hlb2 kernel:  [<c0264979>] ip_rcv+0x485/0x4bd
Jul 26 08:56:26 hlb2 kernel:  [<c026422b>] ip_local_deliver+0x77/0x1e2
Jul 26 08:56:26 hlb2 kernel:  [<e093a7ca>]
e1000_alloc_rx_buffers+0x1d8/0x28d [e1000]
Jul 26 08:56:26 hlb2 kernel:  [<c0264396>]
ip_local_deliver_finish+0x0/0x15e
Jul 26 08:56:26 hlb2 kernel:  [<c024d926>] netif_receive_skb+0x218/0x239
Jul 26 08:56:26 hlb2 kernel:  [<e0939fdb>]
e1000_clean_rx_irq+0x3b7/0x47a [e1000]
Jul 26 08:56:26 hlb2 kernel:  [<c024d926>] netif_receive_skb+0x218/0x239
Jul 26 08:56:26 hlb2 kernel:  [<e093988a>] e1000_clean+0x74/0xf6 [e1000]
Jul 26 08:56:26 hlb2 kernel:  [<c024dac0>] net_rx_action+0x8f/0x13b
Jul 26 08:56:26 hlb2 kernel:  [<c011d68c>] __do_softirq+0x63/0xc1
Jul 26 08:56:26 hlb2 kernel:  [<c011d71b>] do_softirq+0x31/0x35
Jul 26 08:56:26 hlb2 kernel:  [<c0105242>] do_IRQ+0x80/0x93
Jul 26 08:56:26 hlb2 kernel:  [<c02a33e2>] _spin_unlock_irq+0x5/0x7
Jul 26 08:56:26 hlb2 kernel:  [<c02a2209>]
__sched_text_start+0x609/0x679
Jul 26 08:56:26 hlb2 kernel:  [<c0103a13>] common_interrupt+0x23/0x28
Jul 26 08:56:26 hlb2 kernel:  [<c01500d8>] move_vma+0xba/0x1ab
Jul 26 08:56:26 hlb2 kernel:  [<c0101577>]
mwait_idle_with_hints+0x34/0x38
Jul 26 08:56:26 hlb2 kernel:  [<c0101450>] cpu_idle+0x5a/0x6f
Jul 26 08:56:26 hlb2 kernel:  [<c03559e0>] start_kernel+0x1ec/0x1f0
Jul 26 08:56:26 hlb2 kernel:  [<c03553eb>] unknown_bootoption+0x0/0x181
Jul 26 08:56:26 hlb2 kernel:  =======================


Jul 26 08:56:27 hlb2 kernel: WARNING: at net/core/dev.c:1286
skb_gso_segment()
Jul 26 08:56:27 hlb2 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
Jul 26 08:56:27 hlb2 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
Jul 26 08:56:27 hlb2 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
Jul 26 08:56:27 hlb2 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
Jul 26 08:56:27 hlb2 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 08:56:27 hlb2 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 26 08:56:27 hlb2 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
Jul 26 08:56:27 hlb2 kernel:  [<c0114a46>] default_wake_function+0x0/0xc
Jul 26 08:56:27 hlb2 kernel:  [<e084f31f>]
bond_dev_queue_xmit+0x19c/0x1a5 [bond0]
Jul 26 08:56:27 hlb2 kernel:  [<e0855f9d>] bond_3ad_xmit_xor+0x152/0x15c
[bond0]
Jul 26 08:56:27 hlb2 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
Jul 26 08:56:27 hlb2 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
Jul 26 08:56:27 hlb2 kernel:  [<c0277f91>] tcp_write_xmit+0x1cc/0x22e
Jul 26 08:56:27 hlb2 kernel:  [<c0278004>]
__tcp_push_pending_frames+0x11/0x60
Jul 26 08:56:27 hlb2 kernel:  [<c02754d8>]
tcp_rcv_established+0xf0/0x59e
Jul 26 08:56:27 hlb2 kernel:  [<e0b782bb>] ip_vs_in+0xa7/0x204 [ip_vs]
Jul 26 08:56:27 hlb2 kernel:  [<c027c1f8>] tcp_v4_do_rcv+0x22/0xbe
Jul 26 08:56:27 hlb2 kernel:  [<c027c71e>] tcp_v4_rcv+0x48a/0x74c
Jul 26 08:56:27 hlb2 kernel:  [<c02642ed>] ip_local_deliver+0x139/0x1e2
Jul 26 08:56:27 hlb2 kernel:  [<c0264396>]
ip_local_deliver_finish+0x0/0x15e
Jul 26 08:56:27 hlb2 kernel:  [<c0264979>] ip_rcv+0x485/0x4bd
Jul 26 08:56:27 hlb2 kernel:  [<c024d926>] netif_receive_skb+0x218/0x239
Jul 26 08:56:27 hlb2 kernel:  [<e0939fdb>]
e1000_clean_rx_irq+0x3b7/0x47a [e1000]
Jul 26 08:56:27 hlb2 kernel:  [<e093988a>] e1000_clean+0x74/0xf6 [e1000]
Jul 26 08:56:27 hlb2 kernel:  [<c024dac0>] net_rx_action+0x8f/0x13b
Jul 26 08:56:27 hlb2 kernel:  [<c011d68c>] __do_softirq+0x63/0xc1
Jul 26 08:56:27 hlb2 kernel:  [<c011d71b>] do_softirq+0x31/0x35
Jul 26 08:56:27 hlb2 kernel:  [<c0105242>] do_IRQ+0x80/0x93
Jul 26 08:56:27 hlb2 kernel:  [<c0103a13>] common_interrupt+0x23/0x28
Jul 26 08:56:27 hlb2 kernel:  [<c02a33db>]
_spin_unlock_irqrestore+0xa/0xc
Jul 26 08:56:27 hlb2 kernel:  [<c01e04fa>] tty_read+0x81/0xaf
Jul 26 08:56:27 hlb2 kernel:  [<c015bec9>] vfs_read+0xb1/0x131
Jul 26 08:56:27 hlb2 kernel:  [<c015c1b9>] sys_read+0x41/0x67
Jul 26 08:56:27 hlb2 kernel:  [<c0103002>] sysenter_past_esp+0x5f/0x85
Jul 26 08:56:27 hlb2 kernel:  =======================


Howard
Classification:  UNCLASSIFIED 
Caveats: NONE
Comment 5 Howard Kash 2007-07-26 06:32:21 UTC 
Classification:  UNCLASSIFIED 
Caveats: NONE

 
Here are a couple of tracebacks that don't even have ip_vs in them.

Jul 26 09:19:20 hlb1 kernel: WARNING: at net/core/dev.c:1286
skb_gso_segment()
Jul 26 09:19:20 hlb1 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
Jul 26 09:19:20 hlb1 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
Jul 26 09:19:20 hlb1 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
Jul 26 09:19:20 hlb1 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
Jul 26 09:19:20 hlb1 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 09:19:20 hlb1 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 26 09:19:20 hlb1 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
Jul 26 09:19:20 hlb1 kernel:  [<c014463e>] __rmqueue+0x7d/0xcd
Jul 26 09:19:20 hlb1 kernel:  [<c014497f>] buffered_rmqueue+0xf3/0x10b
Jul 26 09:19:20 hlb1 kernel:  [<c0276547>] tcp_cwnd_restart+0x17/0xad
Jul 26 09:19:20 hlb1 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
Jul 26 09:19:20 hlb1 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
Jul 26 09:19:20 hlb1 kernel:  [<c0278105>] tcp_push_one+0xb2/0xd3
Jul 26 09:19:20 hlb1 kernel:  [<c026e2ca>] tcp_sendmsg+0x879/0xa95
Jul 26 09:19:20 hlb1 kernel:  [<c0287133>] inet_sendmsg+0x39/0x43
Jul 26 09:19:20 hlb1 kernel:  [<c0243be2>] do_sock_write+0xab/0xb2
Jul 26 09:19:20 hlb1 kernel:  [<c0243c39>] sock_aio_write+0x50/0x5c
Jul 26 09:19:20 hlb1 kernel:  [<c015c008>] do_sync_write+0xbf/0xfc
Jul 26 09:19:20 hlb1 kernel:  [<c02a2b70>] mutex_lock+0x13/0x22
Jul 26 09:19:20 hlb1 kernel:  [<c0129232>]
autoremove_wake_function+0x0/0x33
Jul 26 09:19:20 hlb1 kernel:  [<c02472ad>]
sock_common_getsockopt+0x1c/0x21
Jul 26 09:19:20 hlb1 kernel:  [<c0244c49>] sys_getsockopt+0x7d/0x9c
Jul 26 09:19:20 hlb1 kernel:  [<c015c10e>] vfs_write+0xc9/0x133
Jul 26 09:19:20 hlb1 kernel:  [<c015c220>] sys_write+0x41/0x67
Jul 26 09:19:20 hlb1 kernel:  [<c0103002>] sysenter_past_esp+0x5f/0x85

Jul 26 09:19:24 hlb1 kernel: WARNING: at net/core/dev.c:1286
skb_gso_segment()
Jul 26 09:19:24 hlb1 kernel:  [<c024d0c8>] skb_gso_segment+0x92/0x184
Jul 26 09:19:24 hlb1 kernel:  [<c024d224>] dev_gso_segment+0x10/0x34
Jul 26 09:19:24 hlb1 kernel:  [<c024d293>] dev_hard_start_xmit+0x4b/0xb1
Jul 26 09:19:24 hlb1 kernel:  [<c024d46b>] dev_queue_xmit+0x172/0x1fd
Jul 26 09:19:24 hlb1 kernel:  [<c0267828>] ip_output+0x274/0x2ad
Jul 26 09:19:24 hlb1 kernel:  [<c02673a7>] ip_finish_output+0x0/0x20d
Jul 26 09:19:24 hlb1 kernel:  [<c0267bbf>] ip_queue_xmit+0x35e/0x3a6
Jul 26 09:19:24 hlb1 kernel:  [<e0939900>] e1000_clean+0xea/0xf6 [e1000]
Jul 26 09:19:24 hlb1 kernel:  [<c024dac0>] net_rx_action+0x8f/0x13b
Jul 26 09:19:24 hlb1 kernel:  [<c02a336a>] _spin_lock_bh+0x8/0x10
Jul 26 09:19:24 hlb1 kernel:  [<c024713d>] release_sock+0x10/0x52
Jul 26 09:19:24 hlb1 kernel:  [<c02a33e9>] _spin_unlock_bh+0x5/0xd
Jul 26 09:19:24 hlb1 kernel:  [<c026aef9>] do_ip_getsockopt+0x432/0x4d6
Jul 26 09:19:24 hlb1 kernel:  [<c0276547>] tcp_cwnd_restart+0x17/0xad
Jul 26 09:19:24 hlb1 kernel:  [<c0276cbe>] tcp_transmit_skb+0x3e0/0x402
Jul 26 09:19:24 hlb1 kernel:  [<c0277846>] tso_fragment+0x15a/0x194
Jul 26 09:19:24 hlb1 kernel:  [<c0278105>] tcp_push_one+0xb2/0xd3
Jul 26 09:19:24 hlb1 kernel:  [<c026e2ca>] tcp_sendmsg+0x879/0xa95
Jul 26 09:19:24 hlb1 kernel:  [<c0287133>] inet_sendmsg+0x39/0x43
Jul 26 09:19:24 hlb1 kernel:  [<c0243be2>] do_sock_write+0xab/0xb2
Jul 26 09:19:24 hlb1 kernel:  [<c0243c39>] sock_aio_write+0x50/0x5c
Jul 26 09:19:24 hlb1 kernel:  [<c015c008>] do_sync_write+0xbf/0xfc
Jul 26 09:19:24 hlb1 kernel:  [<c02a2b70>] mutex_lock+0x13/0x22
Jul 26 09:19:24 hlb1 kernel:  [<c0129232>]
autoremove_wake_function+0x0/0x33
Jul 26 09:19:24 hlb1 kernel:  [<c02472ad>]
sock_common_getsockopt+0x1c/0x21
Jul 26 09:19:24 hlb1 kernel:  [<c0244c49>] sys_getsockopt+0x7d/0x9c
Jul 26 09:19:24 hlb1 kernel:  [<c015c10e>] vfs_write+0xc9/0x133
Jul 26 09:19:24 hlb1 kernel:  [<c015c220>] sys_write+0x41/0x67
Jul 26 09:19:24 hlb1 kernel:  [<c0103002>] sysenter_past_esp+0x5f/0x85
Jul 26 09:19:24 hlb1 kernel:  =======================
Classification:  UNCLASSIFIED 
Caveats: NONE
Comment 6 Herbert Xu 2007-07-28 01:48:36 UTC 
What does ethtool -k bondX say? Thanks.
Comment 7 Howard Kash 2007-07-30 07:17:24 UTC 
Classification:  UNCLASSIFIED 
Caveats: NONE


bond0 consists of three slave interfaces (eth0, eth2, and eth4).  Here's
the ethtool output for each.

# ethtool -k bond0
Offload parameters for bond0:
Cannot get device rx csum settings: Operation not supported
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp segmentation offload: on

# ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on

# ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on

# ethtool -k eth4
Offload parameters for eth4:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
Classification:  UNCLASSIFIED 
Caveats: NONE
Comment 8 Herbert Xu 2007-07-30 09:28:57 UTC 
OK, the bonding device is generating bogus settings.  Will fix.  Thanks.
Comment 9 Herbert Xu 2007-08-07 07:41:20 UTC 
Created attachment 12295 [details]
[NET]: Share correct feature code between bridging and bonding

This patch moves the bridge feature computation to core and uses it for bonding.
Comment 10 Benny Amorsen 2007-08-17 05:10:38 UTC 
Could you make a patch for 2.6.22? Our servers crash due to an unrelated problem when trying to boot 2.6.23rc3, so I cannot test.
Comment 11 Herbert Xu 2007-08-20 23:25:21 UTC 
Created attachment 12465 [details]
[NET]: Share correct feature code between bridging and bonding

Here is the back-port for 2.6.22.
Comment 12 Howard Kash 2007-08-21 06:44:27 UTC 
Just upgraded system to 2.6.22.4 with back ported patch and no more warnings from skb_gso_segment()!!  Thanks!

Will 2.6.23 have the patch included?
Comment 13 Herbert Xu 2007-08-21 06:59:47 UTC 
Yes it will.
Comment 14 Benny Amorsen 2007-08-29 14:19:29 UTC 
Unfortunately 2.6.22.5 didn't have it included... The patch works great though.