Bug 204879
Summary: | "invalid inflight", WARNING: CPU: 1 PID: 5103 at net/ipv4/tcp_output.c:2509 tcp_send_loss_probe.cold.42+0x20/0x2d | ||
---|---|---|---|
Product: | Networking | Reporter: | Roman Mamedov (rm+bko) |
Component: | IPV4 | Assignee: | Stephen Hemminger (stephen) |
Status: | RESOLVED CODE_FIX | ||
Severity: | normal | CC: | eric.dumazet, rsalvaterra |
Priority: | P1 | ||
Hardware: | All | ||
OS: | Linux | ||
Kernel Version: | 4.14.143 | Subsystem: | |
Regression: | No | Bisected commit-id: |
Description
Roman Mamedov
2019-09-16 20:28:15 UTC
Also happening on 4.14.144. Backtrace from OpenWrt (no symbols. though). Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.168863] invalid inflight: 1 state 1 cwnd 10 mss 1448 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.174283] ------------[ cut here ]------------ Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.178925] WARNING: CPU: 1 PID: 0 at 0xffffffff8166702d Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.184256] Modules linked in: nf_nat_pptp nf_conntrack_pptp nf_conntrack_ipv6 lzo lz4 iptable_nat ipt_REJECT ipt_MASQUERADE xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY wireguard ts_fsm ts_bm nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_redirect nf_nat_proto_gre nf_nat_masquerade_ipv4 nf_nat_irc nf_conntrack_ipv4 nf_nat_ipv4 nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_rtcache nf_conntrack_proto_gre nf_conntrack_netlink Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.255920] nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast ts_kmp nf_conntrack_amanda lzo_decompress lzo_compress lz4_decompress lz4_compress iptable_raw iptable_mangle iptable_filter ipt_ECN ip6table_raw ip_tables sch_cake nf_conntrack sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred i2c_dev ledtrig_heartbeat xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.327283] x_tables nf_reject_ipv6 igb ifb ip6_udp_tunnel udp_tunnel zram zsmalloc sha256_ssse3 sha1_ssse3 ccp_crypto rsa_generic mpi asn1_decoder akcipher ccp sha256_generic sha1_generic authenc crypto_acompress sdhci_pltfm leds_apu2 ptp pps_core Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.349437] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.144 #0 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.355555] Hardware name: PC Engines APU2/APU2, BIOS 4.0.7 02/28/2017 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.362092] task: ffff88811a08f300 task.stack: ffffc90000060000 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.368029] RIP: 0010:0xffffffff8166702d Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.371970] RSP: 0018:ffff88811ec83e70 EFLAGS: 00010282 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.377213] RAX: 000000000000002c RBX: ffff8881170a9f00 RCX: 0000000000000000 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.384358] RDX: ffff88811ec9ba60 RSI: ffff88811ec96358 RDI: ffff88811ec96358 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.391532] RBP: ffff88811ec83e88 R08: 0000000000000000 R09: 00000000000001fd Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.398690] R10: ffff88811ec83e68 R11: 0000000000000000 R12: ffff8881170aa050 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.405866] R13: ffff8881170aa050 R14: ffff88811ec83ef0 R15: ffff88811ec9a4a8 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.413029] FS: 0000000000000000(0000) GS:ffff88811ec80000(0000) knlGS:0000000000000000 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.421132] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.426892] CR2: 00000000007a3060 CR3: 00000001162ce000 CR4: 00000000000406e0 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.434042] Call Trace: Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.436513] <IRQ> Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.438549] 0xffffffff81668cf5 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.441712] 0xffffffff81668e9b Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.444872] ? 0xffffffff81668e70 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.448200] 0xffffffff810ca69f Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.451364] 0xffffffff810ca834 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.454520] ? 0xffffffff81040d3e Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.457846] ? 0xffffffff810d5d60 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.461176] 0xffffffff81a000ba Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.464330] 0xffffffff81087e52 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.467485] 0xffffffff818024d7 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.470640] 0xffffffff81801aec Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.473794] </IRQ> Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.475907] RIP: 0010:0xffffffff815b3a86 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.479842] RSP: 0018:ffffc90000063e98 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.487432] RAX: ffff88811ec9f340 RBX: 0000000000000002 RCX: 000000000000001f Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.494587] RDX: 0000000d13ec8012 RSI: 00000000803d7d59 RDI: 0000000000000000 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.501737] RBP: ffffc90000063ec8 R08: 0000000000000004 R09: 000000000001eb80 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.508888] R10: ffffc90000063e68 R11: 0000000000000000 R12: ffff888119a7da00 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.516047] R13: 0000000d13ec8012 R14: ffffffff81e459b8 R15: 0000000000000000 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.523202] ? 0xffffffff815b3a5e Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.526531] 0xffffffff815b3b32 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.529685] 0xffffffff810b1443 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.532838] 0xffffffff810b15e0 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.535995] 0xffffffff810b17bf Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.539149] 0xffffffff8103f8f0 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.542313] 0xffffffff810000d5 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.545468] Code: 74 2b 80 3d e6 c1 81 00 00 75 22 0f b6 53 12 8b 8b 14 06 00 00 41 89 c0 48 c7 c7 b0 03 d4 81 c6 05 c9 c1 81 00 01 e8 ef 30 a5 ff <0f> 0b c6 83 62 04 00 00 00 e9 ce 00 00 00 8b 93 98 05 00 00 85 Fri Sep 20 19:23:29 2019 kern.warn kernel: [ 56.564513] ---[ end trace a7d4411530d0c7d2 ]--- Sorry, my previous trace was from the log dump. Here's directy from dmesg: [ 56.168863] invalid inflight: 1 state 1 cwnd 10 mss 1448 [ 56.174283] ------------[ cut here ]------------ [ 56.178925] WARNING: CPU: 1 PID: 0 at 0xffffffff8166702d [ 56.184256] Modules linked in: nf_nat_pptp nf_conntrack_pptp nf_conntrack_ipv6 lzo lz4 iptable_nat ipt_REJECT ipt_MASQUERADE xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY wireguard ts_fsm ts_bm nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_redirect nf_nat_proto_gre nf_nat_masquerade_ipv4 nf_nat_irc nf_conntrack_ipv4 nf_nat_ipv4 nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_rtcache nf_conntrack_proto_gre nf_conntrack_netlink [ 56.255920] nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast ts_kmp nf_conntrack_amanda lzo_decompress lzo_compress lz4_decompress lz4_compress iptable_raw iptable_mangle iptable_filter ipt_ECN ip6table_raw ip_tables sch_cake nf_conntrack sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred i2c_dev ledtrig_heartbeat xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT [ 56.327283] x_tables nf_reject_ipv6 igb ifb ip6_udp_tunnel udp_tunnel zram zsmalloc sha256_ssse3 sha1_ssse3 ccp_crypto rsa_generic mpi asn1_decoder akcipher ccp sha256_generic sha1_generic authenc crypto_acompress sdhci_pltfm leds_apu2 ptp pps_core [ 56.349437] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.144 #0 [ 56.355555] Hardware name: PC Engines APU2/APU2, BIOS 4.0.7 02/28/2017 [ 56.362092] task: ffff88811a08f300 task.stack: ffffc90000060000 [ 56.368029] RIP: 0010:0xffffffff8166702d [ 56.371970] RSP: 0018:ffff88811ec83e70 EFLAGS: 00010282 [ 56.377213] RAX: 000000000000002c RBX: ffff8881170a9f00 RCX: 0000000000000000 [ 56.384358] RDX: ffff88811ec9ba60 RSI: ffff88811ec96358 RDI: ffff88811ec96358 [ 56.391532] RBP: ffff88811ec83e88 R08: 0000000000000000 R09: 00000000000001fd [ 56.398690] R10: ffff88811ec83e68 R11: 0000000000000000 R12: ffff8881170aa050 [ 56.405866] R13: ffff8881170aa050 R14: ffff88811ec83ef0 R15: ffff88811ec9a4a8 [ 56.413029] FS: 0000000000000000(0000) GS:ffff88811ec80000(0000) knlGS:0000000000000000 [ 56.421132] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 56.426892] CR2: 00000000007a3060 CR3: 00000001162ce000 CR4: 00000000000406e0 [ 56.434042] Call Trace: [ 56.436513] <IRQ> [ 56.438549] 0xffffffff81668cf5 [ 56.441712] 0xffffffff81668e9b [ 56.444872] ? 0xffffffff81668e70 [ 56.448200] 0xffffffff810ca69f [ 56.451364] 0xffffffff810ca834 [ 56.454520] ? 0xffffffff81040d3e [ 56.457846] ? 0xffffffff810d5d60 [ 56.461176] 0xffffffff81a000ba [ 56.464330] 0xffffffff81087e52 [ 56.467485] 0xffffffff818024d7 [ 56.470640] 0xffffffff81801aec [ 56.473794] </IRQ> [ 56.475907] RIP: 0010:0xffffffff815b3a86 [ 56.479842] RSP: 0018:ffffc90000063e98 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10 [ 56.487432] RAX: ffff88811ec9f340 RBX: 0000000000000002 RCX: 000000000000001f [ 56.494587] RDX: 0000000d13ec8012 RSI: 00000000803d7d59 RDI: 0000000000000000 [ 56.501737] RBP: ffffc90000063ec8 R08: 0000000000000004 R09: 000000000001eb80 [ 56.508888] R10: ffffc90000063e68 R11: 0000000000000000 R12: ffff888119a7da00 [ 56.516047] R13: 0000000d13ec8012 R14: ffffffff81e459b8 R15: 0000000000000000 [ 56.523202] ? 0xffffffff815b3a5e [ 56.526531] 0xffffffff815b3b32 [ 56.529685] 0xffffffff810b1443 [ 56.532838] 0xffffffff810b15e0 [ 56.535995] 0xffffffff810b17bf [ 56.539149] 0xffffffff8103f8f0 [ 56.542313] 0xffffffff810000d5 [ 56.545468] Code: 74 2b 80 3d e6 c1 81 00 00 75 22 0f b6 53 12 8b 8b 14 06 00 00 41 89 c0 48 c7 c7 b0 03 d4 81 c6 05 c9 c1 81 00 01 e8 ef 30 a5 ff <0f> 0b c6 83 62 04 00 00 00 e9 ce 00 00 00 8b 93 98 05 00 00 85 [ 56.564513] ---[ end trace a7d4411530d0c7d2 ]--- This might be fixed in 4.14.146 ba2ddb43f270e6492ccce4fc42fc32c611de8f68 tcp: Don't dequeue SYN/FIN-segments from write-queue f1dcc5ed4bea3f2d63b74ad86617ec12b1e5e9d4 tcp: Reset send_head when removing skb from write-queue Yes I don't get these anymore recently, thanks. |