Bug 8338 - NAT of TCP connections broken
Summary: NAT of TCP connections broken
Status: REJECTED UNREPRODUCIBLE
Alias: None
Product: Networking
Classification: Unclassified
Component: Netfilter/Iptables (show other bugs)
Hardware: i386 Linux
: P2 high
Assignee: networking_netfilter-iptables@kernel-bugs.osdl.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-04-16 13:35 UTC by Ernestas Vaiciukevičius
Modified: 2008-09-26 05:53 UTC (History)
2 users (show)

See Also:
Kernel Version: 2.6.20.7
Subsystem:
Regression: ---
Bisected commit-id:


Attachments
tcpdump log (3.05 KB, text/plain)
2007-04-16 13:42 UTC, Ernestas Vaiciukevičius
Details
kernel config (62.21 KB, text/plain)
2007-04-16 13:44 UTC, Ernestas Vaiciukevičius
Details
external interface dump (13.16 KB, text/plain)
2007-04-17 10:47 UTC, Ernestas Vaiciukevičius
Details
internal interface (4.18 KB, text/plain)
2007-04-17 10:48 UTC, Ernestas Vaiciukevičius
Details
wireshark log on external interface (37.48 KB, application/octet-stream)
2007-04-19 10:22 UTC, Ernestas Vaiciukevičius
Details
wireshark log on internal interface (6.79 KB, application/octet-stream)
2007-04-19 10:23 UTC, Ernestas Vaiciukevičius
Details

Description Ernestas Vaiciukevičius 2007-04-16 13:35:43 UTC
Most recent kernel where this bug did *NOT* occur: 2.6.18
Distribution: Debian
Hardware Environment: amd64 4200x2
Software Environment:
Problem Description:
I'm using my debian box for sharing internet connection. After update to 
debian's 2.6.20 kernel (and adjusting new kernel configuration for NAT) NAT of 
TCP connections stoped working, while ICMP and UDP is working (ping, traceroute 
and DNS). When trying to open web page on a PC from my LAN (tried with linux 
and winxp), it sends the request but is unable to receive correct response from 
any site. On the router PC www works ok. 

I tried both new "Independent connection tracking" as well as "Dependent 
(obsolete) connection tracking" options with no luck.

tcp_window_scaling setting seems to have no effect.
Comment 1 Ernestas Vaiciukevičius 2007-04-16 13:42:13 UTC
Created attachment 11159 [details]
tcpdump log

tcpdump log of unsuccessful connection to www.google.lt
(it can be incomplete-I can make more tests)
Comment 2 Ernestas Vaiciukevičius 2007-04-16 13:44:20 UTC
Created attachment 11160 [details]
kernel config

my kernel config
Comment 3 Anonymous Emailer 2007-04-16 13:48:29 UTC
Reply-To: akpm@linux-foundation.org

> On Mon, 16 Apr 2007 13:35:47 -0700 bugme-daemon@bugzilla.kernel.org wrote:
> http://bugzilla.kernel.org/show_bug.cgi?id=8338
> 
>            Summary: NAT of TCP connections broken
>     Kernel Version: 2.6.20.7
>             Status: NEW
>           Severity: high
>              Owner: networking_netfilter-iptables@kernel-bugs.osdl.org
>          Submitter: ernisv@gmail.com
> 
> 
> Most recent kernel where this bug did *NOT* occur: 2.6.18
> Distribution: Debian
> Hardware Environment: amd64 4200x2
> Software Environment:
> Problem Description:
> I'm using my debian box for sharing internet connection. After update to 
> debian's 2.6.20 kernel (and adjusting new kernel configuration for NAT) NAT of 
> TCP connections stoped working, while ICMP and UDP is working (ping, traceroute 
> and DNS). When trying to open web page on a PC from my LAN (tried with linux 
> and winxp), it sends the request but is unable to receive correct response from 
> any site. On the router PC www works ok. 
> 
> I tried both new "Independent connection tracking" as well as "Dependent 
> (obsolete) connection tracking" options with no luck.
> 
> tcp_window_scaling setting seems to have no effect.
> 

Comment 4 Patrick McHardy 2007-04-17 01:02:33 UTC
Please do "echo 255 > /proc/sys/net/ipv4/netfilter/ip_conntrack_log_invalid" and
send me the resulting log output.
Comment 5 Patrick McHardy 2007-04-17 06:01:31 UTC
Andrew Morton wrote:
>>On Mon, 16 Apr 2007 13:35:47 -0700 bugme-daemon@bugzilla.kernel.org wrote:
>>http://bugzilla.kernel.org/show_bug.cgi?id=8338
>>
>>I'm using my debian box for sharing internet connection. After update to 
>>debian's 2.6.20 kernel (and adjusting new kernel configuration for NAT) NAT of 
>>TCP connections stoped working, while ICMP and UDP is working (ping, traceroute 
>>and DNS). When trying to open web page on a PC from my LAN (tried with linux 
>>and winxp), it sends the request but is unable to receive correct response from 
>>any site. On the router PC www works ok. 
>>
>>I tried both new "Independent connection tracking" as well as "Dependent 
>>(obsolete) connection tracking" options with no luck.
>>
>>tcp_window_scaling setting seems to have no effect.


Please send packet dumps from both the incoming and outgoing interfaces.

Comment 6 Ernestas Vaiciukevičius 2007-04-17 10:47:20 UTC
Created attachment 11219 [details]
external interface dump

dump of traffic on external interface
Comment 7 Ernestas Vaiciukevičius 2007-04-17 10:48:26 UTC
Created attachment 11220 [details]
internal interface

dump of traffic on internal interface
Comment 8 Ernestas Vaiciukevičius 2007-04-17 10:52:04 UTC
before testing I executed "echo 255 
> /proc/sys/net/ipv4/netfilter/ip_conntrack_log_invalid", 
but there were no error messages in the logs.
Comment 9 Patrick McHardy 2007-04-17 21:30:57 UTC
All I can see is a connection that is properly NATed and seems to work perfectly
fine.
Comment 10 Ernestas Vaiciukevičius 2007-04-17 22:45:07 UTC
All browsers I tried on LAN PC's do not open any sites. They keep waiting for
the response ("Request sent") and then timeouts.
They can connect to the router machine via ssh and show correctly the www hosted
on the router (when connection is not NAT'ed).
For now I'm still using kernel 2.6.18 - with it everything works fine with the
same iptables configuration.

I tried to watch the traffic on internal interface with wireshark and IIRC it
showed some invalid packets.
I could repeat that and send you the log of wireshark today evening.
Comment 11 Ernestas Vaiciukevičius 2007-04-19 10:22:22 UTC
Created attachment 11236 [details]
wireshark log on external interface

wireshark log made on external interface
Comment 12 Ernestas Vaiciukevičius 2007-04-19 10:23:34 UTC
Created attachment 11237 [details]
wireshark log on internal interface

wireshark log made on internal (LAN) interface
Comment 13 Ernestas Vaiciukevičius 2007-04-19 10:29:02 UTC
I've uploaded the log's on both interfaces made with wireshark (former 
ethereal). Wireshark shows some lost/invalid pakets in red on both interfaces 
when trying to connect to www.google.lt from LAN pc. Also there are 
suspicious 'Ethernet II' type packets with HTML data - most likely they are the 
missing/corrupt TCP packets.

Again when accessing the web pages from router PC - everything is fine, 
wireshark shows correct streams..
Comment 14 Beau V.C. Bellamy 2007-04-24 04:20:05 UTC
I have this very same problem.  A 64 bit stock 2.6.20.7 kernel.  
The "Independent connection tracking" option is configured.   Based on 
tethereal dumps,  It seems like some packets are getting through initially,  
then it just stops forwarding packets all together.  When persistently 
restarting a cvs pserver connection or a HTTP request, it will stop at 
different places each time.  Connections initiated from the firewall itself are 
fine.
Comment 15 Ernestas Vaiciukevičius 2007-04-24 05:20:50 UTC
I forgot to mention - I'm also using 64-bit kernel. So perhaps it's 64-bit specific?
Comment 16 Patrick McHardy 2007-04-24 05:36:15 UTC
@Ernestas:

Looking at the dumps, I still fail to see any sign of broken NAT. A lost packet
is not an error, it can happen for any number of reasons. In fact you can see
multiple connections that are properly NATed.

What can also be seen is your firewall sending a RST in response to a
retransmission. The reason for this might be that the retransmission is regarded
as out-of-window by TCP connection tracking and is therefore not tracked and
NATed back, so it goes to the firewall itself, where the connection is unknown
and thus reset. If this is the case, "iptables -t mangle -A PREROUTING -m state
--state INVALID -j DROP" should cure the problem.
Comment 17 Ernestas Vaiciukevičius 2007-04-24 12:49:20 UTC
I've tried the rule
"iptables -t mangle -A PREROUTING -m state --state INVALID -j DROP"

It doesn't help, but there are packets matching the rule. I've logged them 
using "iptables -t mangle -A PREROUTING -m state --state INVALID -j LOG":

Apr 24 22:07:18 gw vmunix: IN=eth1 OUT= MAC=*********** SRC=209.85.129.104 
DST=******** LEN=40 TOS=0x00 PREC=0x00 TTL=246 ID=53516 PROTO=TCP SPT=80 
DPT=1067 WINDOW=8190 RES=0x00 ACK FIN URGP=0 
Apr 24 22:07:18 gw vmunix: IN=eth1 OUT= MAC=*********** SRC=209.85.129.104 
DST=******** LEN=40 TOS=0x00 PREC=0x00 TTL=246 ID=21773 PROTO=TCP SPT=80 
DPT=1065 WINDOW=8190 RES=0x00 ACK FIN URGP=0 
Apr 24 22:07:19 gw vmunix: IN=eth1 OUT= MAC=*********** SRC=209.85.129.104 
DST=******** LEN=40 TOS=0x00 PREC=0x00 TTL=246 ID=44051 PROTO=TCP SPT=80 
DPT=1067 WINDOW=8190 RES=0x00 ACK FIN URGP=0 
Apr 24 22:07:19 gw vmunix: IN=eth1 OUT= MAC=*********** SRC=209.85.129.104 
DST=******** LEN=40 TOS=0x00 PREC=0x00 TTL=246 ID=32547 PROTO=TCP SPT=80 
DPT=1065 WINDOW=8190 RES=0x00 ACK FIN URGP=0 
Apr 24 22:07:21 gw vmunix: IN=eth1 OUT= MAC=*********** SRC=209.85.129.104 
DST=******** LEN=40 TOS=0x00 PREC=0x00 TTL=246 ID=38593 PROTO=TCP SPT=80 
DPT=1067 WINDOW=8190 RES=0x00 ACK FIN URGP=0 
Apr 24 22:07:21 gw vmunix: IN=eth1 OUT= MAC=*********** SRC=209.85.129.104 
DST=******** LEN=40 TOS=0x00 PREC=0x00 TTL=246 ID=3562 PROTO=TCP SPT=80 
DPT=1065 WINDOW=8190 RES=0x00 ACK FIN URGP=0

Also I'm not sure why wireshark shows some 'Ethernet II' type packets (why they 
are not recognized as TCP) with html data in them in the previous logs.
Comment 18 Beau V.C. Bellamy 2007-04-25 11:52:17 UTC
So is there anything that 'I' can do to help track this down?   I believe that 
this is a very real regression that appeared around the 2.6.20 time.  NAT rules 
that I've been using for years are no longer working under this 64 bit kernel.  
Even though the new connection tracking options are configured.  (I've tried 
both the new and old with no effect)
Comment 19 Patrick McHardy 2007-04-26 10:03:53 UTC
For a start a tcpdump that actually shows the problem, including information
which connection is affected in case there's other stuff in it, would be helpful.
Comment 20 Beau V.C. Bellamy 2007-04-28 04:10:49 UTC
Ok...  After failing to see errors in the ethereal dumps.  I started to 
investigate other things and I think I narrowed the problem down.  I believe 
there is a problem with the MTU path resolution.   I use an ADSL connection and 
hence my external MTU is at 1492.   Once I drop the MTU size on the computers 
inside my network to that value,  they are able to transfer through the nat 
fine.  Is this expected behavior in the new 2.6.20+ kernels?    I've changed 
nothing, except going to a 64bit 2.6.20.7 kernel from a 32bit 2.6.19 one.  The 
iptables rules are the same.
Comment 21 Ernestas Vaiciukevičius 2007-04-29 03:33:08 UTC
I can confirm that - when I lower MTU on my LAN PC to 579, everything starts to 
work.
I'm connected to the internet through a modem connected to a cable TV cable - 
don't know how this technology is called :)

I'm also using the same iptables config, etc. and with 2.6.18 64-bit kernel 
everything works ok.
Comment 22 Patrick McHardy 2007-04-29 05:48:14 UTC
Please capture a SYN packet with MSS option on both a working and non-working
kernel and post it here. BTW, is either one of you using the TCPMSS target?
Comment 23 Ernestas Vaiciukevičius 2007-04-29 07:17:29 UTC
Here's another working/non-working dumps. The only difference I see is 
the 'win=0' of the last few packets.

Working 2.6.18 kernel:

ernis@gw:~$ sudo tcpdump -i eth1 -vv dst port 80 or src port 80
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
17:07:51.659411 IP (tos 0x0, ttl  63, id 25909, offset 0, flags [DF], proto: 
TCP (6), length: 60) ctv-213-164-110-227.vinita.lt.2232 > 
nf-in-f147.google.com.www: S, cksum 0x9c2e (correct), 3894863084:3894863084(0) 
win 5840 <mss 1460,sackOK,timestamp 819673 0,nop,wscale 3>
17:07:51.717877 IP (tos 0x0, ttl 246, id 35481, offset 0, flags [none], proto: 
TCP (6), length: 44) nf-in-f147.google.com.www > 
ctv-213-164-110-227.vinita.lt.2232: S, cksum 0x8e49 (correct), 
2154387012:2154387012(0) ack 3894863085 win 8190 <mss 1460>
17:07:51.719639 IP (tos 0x0, ttl  63, id 25910, offset 0, flags [DF], proto: 
TCP (6), length: 40) ctv-213-164-110-227.vinita.lt.2232 > 
nf-in-f147.google.com.www: ., cksum 0xaf34 (correct), 1:1(0) ack 1 win 5840
17:07:51.722376 IP (tos 0x0, ttl  63, id 25911, offset 0, flags [DF], proto: 
TCP (6), length: 229) ctv-213-164-110-227.vinita.lt.2232 > 
nf-in-f147.google.com.www: P 1:190(189) ack 1 win 5840
17:07:51.793876 IP (tos 0x0, ttl  55, id 34896, offset 0, flags [none], proto: 
TCP (6), length: 40) nf-in-f147.google.com.www > 
ctv-213-164-110-227.vinita.lt.2232: ., cksum 0xac27 (correct), 1:1(0) ack 190 
win 6432
17:07:51.823877 IP (tos 0x0, ttl  55, id 34898, offset 0, flags [none], proto: 
TCP (6), length: 1470) nf-in-f147.google.com.www > 
ctv-213-164-110-227.vinita.lt.2232: . 1:1431(1430) ack 190 win 6432
17:07:51.825895 IP (tos 0x0, ttl  63, id 25912, offset 0, flags [DF], proto: 
TCP (6), length: 40) ctv-213-164-110-227.vinita.lt.2232 > 
nf-in-f147.google.com.www: ., cksum 0x9e2d (correct), 190:190(0) ack 1431 win 
8580
17:07:51.841877 IP (tos 0x0, ttl  55, id 34900, offset 0, flags [none], proto: 
TCP (6), length: 1470) nf-in-f147.google.com.www > 
ctv-213-164-110-227.vinita.lt.2232: . 1431:2861(1430) ack 190 win 6432
17:07:51.844253 IP (tos 0x0, ttl  63, id 25913, offset 0, flags [DF], proto: 
TCP (6), length: 40) ctv-213-164-110-227.vinita.lt.2232 > 
nf-in-f147.google.com.www: ., cksum 0x8d6b (correct), 190:190(0) ack 2861 win 
11440
17:07:51.859893 IP (tos 0x0, ttl  55, id 34902, offset 0, flags [none], proto: 
TCP (6), length: 368) nf-in-f147.google.com.www > 
ctv-213-164-110-227.vinita.lt.2232: P 2861:3189(328) ack 190 win 6432
17:07:51.861528 IP (tos 0x0, ttl  63, id 25914, offset 0, flags [DF], proto: 
TCP (6), length: 40) ctv-213-164-110-227.vinita.lt.2232 > 
nf-in-f147.google.com.www: ., cksum 0x80f7 (correct), 190:190(0) ack 3189 win 
14300

Not-working 2.6.20 kernel:

17:04:02.252563 IP (tos 0x0, ttl  63, id 56964, offset 0, flags [DF], proto: 
TCP (6), length: 60) ctv-213-164-110-227.vinita.lt.2191 > 
nf-in-f99.google.com.www: S, cksum 0x3174 (correct), 3647528203:3647528203(0) 
win 5840 <mss 1460,sackOK,timestamp 590735 0,nop,wscale 3>
17:04:02.316573 IP (tos 0x0, ttl 246, id 33274, offset 0, flags [none], proto: 
TCP (6), length: 44) nf-in-f99.google.com.www > 
ctv-213-164-110-227.vinita.lt.2191: S, cksum 0x0f85 (correct), 
338647099:338647099(0) ack 3647528204 win 8190 <mss 1460>
17:04:02.317948 IP (tos 0x0, ttl  63, id 56965, offset 0, flags [DF], proto: 
TCP (6), length: 40) ctv-213-164-110-227.vinita.lt.2191 > 
nf-in-f99.google.com.www: ., cksum 0x3070 (correct), 1:1(0) ack 1 win 5840
17:04:02.320576 IP (tos 0x0, ttl  63, id 56966, offset 0, flags [DF], proto: 
TCP (6), length: 229) ctv-213-164-110-227.vinita.lt.2191 > 
nf-in-f99.google.com.www: P 1:190(189) ack 1 win 5840
17:04:02.386573 IP (tos 0x0, ttl  55, id 54644, offset 0, flags [none], proto: 
TCP (6), length: 40) nf-in-f99.google.com.www > 
ctv-213-164-110-227.vinita.lt.2191: ., cksum 0x2d63 (correct), 1:1(0) ack 190 
win 6432
17:04:02.452574 IP (tos 0x0, ttl  55, id 54650, offset 0, flags [none], proto: 
TCP (6), length: 368) nf-in-f99.google.com.www > 
ctv-213-164-110-227.vinita.lt.2191: P 2861:3189(328) ack 190 win 6432
17:04:02.454324 IP (tos 0x0, ttl  63, id 56967, offset 0, flags [DF], proto: 
TCP (6), length: 40) ctv-213-164-110-227.vinita.lt.2191 > 
nf-in-f99.google.com.www: ., cksum 0x2fb3 (correct), 190:190(0) ack 1 win 5840
17:04:05.487309 IP (tos 0x0, ttl  63, id 56968, offset 0, flags [DF], proto: 
TCP (6), length: 40) ctv-213-164-110-227.vinita.lt.2191 > 
nf-in-f99.google.com.www: F, cksum 0x2fb2 (correct), 190:190(0) ack 1 win 5840
17:04:05.571674 IP (tos 0x0, ttl  55, id 54708, offset 0, flags [none], proto: 
TCP (6), length: 368) nf-in-f99.google.com.www > 
ctv-213-164-110-227.vinita.lt.2191: P 2861:3189(328) ack 190 win 6432
17:04:05.573800 IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 40) ctv-213-164-110-227.vinita.lt.2191 > nf-in-f99.google.com.www: 
R, cksum 0xb2fa (correct), 3647528393:3647528393(0) win 0
17:04:05.575674 IP (tos 0x0, ttl  55, id 39735, offset 0, flags [none], proto: 
TCP (6), length: 40) nf-in-f99.google.com.www > 
ctv-213-164-110-227.vinita.lt.2191: ., cksum 0x20ee (correct), 3189:3189(0) ack 
191 win 6432
17:04:05.577298 IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 40) ctv-213-164-110-227.vinita.lt.2191 > nf-in-f99.google.com.www: 
R, cksum 0xb2f9 (correct), 3647528394:3647528394(0) win 0

MTU on router's external interface eth1 is 576.
And no - I'm not using TCPMSS.
Comment 24 Patrick McHardy 2007-05-29 04:33:34 UTC
It certainly doesn't look like it has anything to do with NAT.

WRT path MTU discovery, could you run

"tcpdump -i any -ntv icmp -s0"

on the router with both the broken and working kernel and post the output?
Comment 25 Natalie Protasevich 2007-07-07 16:07:42 UTC
Ernestas, Beau, were you able to collect dumps requested in #24 to help Patrick in tracking this down?
Thanks.
Comment 26 Ernestas Vaiciukevi&#269;ius 2007-07-09 14:24:11 UTC
I'm still using the 2.6.18 kernel for several other reasons. I'll try to find the time to collect the dumps and try the 2.6.21 kernel this week..

(In reply to comment #25)
> Ernestas, Beau, were you able to collect dumps requested in #24 to help
> Patrick
> in tracking this down?
> Thanks.
> 
Comment 27 Ernestas Vaiciukevi&#269;ius 2007-07-10 10:46:24 UTC
This is a dump from internal router interface (vanilla kernel 2.6.22 - still not working):
sudo tcpdump -i eth0 -ntv icmp
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
IP (tos 0xc0, ttl  64, id 21467, offset 0, flags [none], proto: ICMP (1), length: 400) 5.1.1.5 > 5.1.1.2: ICMP 206.82.140.163 unreachable - need to frag (mtu 576), length 380
        IP (tos 0x0, ttl 128, id 22, offset 0, flags [DF], proto: TCP (6), length: 959) 5.1.1.2.1031 > 206.82.140.163.80: P 106783107:106784026(919) ack 2999603564 win 65535[|icmp]

and this from external router interface:

sudo tcpdump -i eth1  -ntv icmp -s0
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
IP (tos 0xc0, ttl  64, id 13339, offset 0, flags [none], proto: ICMP (1), length: 468) 213.164.99.76 > 213.197.187.205: ICMP 213.164.99.76 unreachable - need to frag (mtu 400), length 448
        IP (tos 0x0, ttl  58, id 1841, offset 0, flags [DF], proto: TCP (6), length: 440) 213.197.187.205.80 > 213.164.99.76.1035: P, cksum 0x8961 (correct), 3909927379:3909927779(400) ack 1154770774 win 6432
        MPLS extension v6 packet not supported
IP (tos 0xc0, ttl  64, id 13340, offset 0, flags [none], proto: ICMP (1), length: 468) 213.164.99.76 > 213.197.187.205: ICMP 213.164.99.76 unreachable - need to frag (mtu 400), length 448
        IP (tos 0x0, ttl  58, id 1844, offset 0, flags [DF], proto: TCP (6), length: 440) 213.197.187.205.80 > 213.164.99.76.1035: P, cksum 0x8961 (correct), 0:400(400) ack 1 win 6432
        MPLS extension v6 packet not supported
IP (tos 0xc0, ttl  64, id 13341, offset 0, flags [none], proto: ICMP (1), length: 468) 213.164.99.76 > 213.197.187.205: ICMP 213.164.99.76 unreachable - need to frag (mtu 400), length 448
        IP (tos 0x0, ttl  58, id 1846, offset 0, flags [DF], proto: TCP (6), length: 440) 213.197.187.205.80 > 213.164.99.76.1035: P, cksum 0x8961 (correct), 0:400(400) ack 1 win 6432
        MPLS extension v6 packet not supported
IP (tos 0xc0, ttl  64, id 13342, offset 0, flags [none], proto: ICMP (1), length: 468) 213.164.99.76 > 213.197.187.205: ICMP 213.164.99.76 unreachable - need to frag (mtu 400), length 448
        IP (tos 0x0, ttl  58, id 1847, offset 0, flags [DF], proto: TCP (6), length: 440) 213.197.187.205.80 > 213.164.99.76.1035: P, cksum 0x8961 (correct), 0:400(400) ack 1 win 6432
        MPLS extension v6 packet not supported
IP (tos 0xc0, ttl  64, id 13343, offset 0, flags [none], proto: ICMP (1), length: 468) 213.164.99.76 > 213.197.187.205: ICMP 213.164.99.76 unreachable - need to frag (mtu 400), length 448
        IP (tos 0x0, ttl  58, id 1849, offset 0, flags [DF], proto: TCP (6), length: 440) 213.197.187.205.80 > 213.164.99.76.1035: P, cksum 0x8960 (correct), 0:400(400) ack 2 win 6432
        MPLS extension v6 packet not supported
IP (tos 0xc0, ttl  64, id 13344, offset 0, flags [none], proto: ICMP (1), length: 468) 213.164.99.76 > 213.197.187.205: ICMP 213.164.99.76 unreachable - need to frag (mtu 400), length 448
        IP (tos 0x0, ttl  58, id 1851, offset 0, flags [DF], proto: TCP (6), length: 440) 213.197.187.205.80 > 213.164.99.76.1035: P, cksum 0x8960 (correct), 0:400(400) ack 2 win 6432
        MPLS extension v6 packet not supported

I tried to open web page from windows pc.
Comment 28 Ernestas Vaiciukevi&#269;ius 2007-07-10 10:56:01 UTC
This is internal internal router interface dump with working kernel (2.6.18).

sudo tcpdump -i eth0 -ntv icmp
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
IP (tos 0xc0, ttl  64, id 5089, offset 0, flags [none], proto: ICMP (1), length: 576) 213.164.99.76 > 5.1.1.2: ICMP 209.85.135.147 unreachable - need to frag (mtu 576), length 556
        IP (tos 0x0, ttl 127, id 103, offset 0, flags [DF], proto: TCP (6), length: 704) 5.1.1.2.1037 > 209.85.135.147.80: P 4069568106:4069568770(664) ack 2139220151 win 65535[|icmp]

On external interface there were no packets.
Comment 29 Ernestas Vaiciukevi&#269;ius 2007-07-10 11:04:16 UTC
For the first (broken kernel) external interface dump:
I think most of these packets were generated because internal interface MTU was set to 400 when I made the dump (I was just experimenting and forgot to restore it)
Comment 30 Ernestas Vaiciukevi&#269;ius 2007-07-11 06:15:27 UTC
I guess it's all right with the ICMP packets/PMTU ?

Router correctly reports that packets should be fragmented, but somehow it still doesn't work with 2.6.22 kernel...
Comment 31 Patrick McHardy 2007-07-11 06:24:30 UTC
Subject: Re:  NAT of TCP connections broken

bugme-daemon@bugzilla.kernel.org wrote:
> http://bugzilla.kernel.org/show_bug.cgi?id=8338
> 
> 
> 
> 
> 
> ------- Comment #30 from ernisv@gmail.com  2007-07-11 06:15 -------
> I guess it's all right with the ICMP packets/PMTU ?
> 
> Router correctly reports that packets should be fragmented, but somehow it
> still doesn't work with 2.6.22 kernel...


I'm not sure which of your dumps is correct and shows the problem,
using a MTU of 400 is expected not to work.

One more thing that might be helpful is if you try to LOG all
-m state --state INVALID packets, there is a bug related to
ICMP NAT that might be related (we're still working on the fix).
Comment 32 Ernestas Vaiciukevi&#269;ius 2007-07-12 08:13:45 UTC
Ok, I tried againg and have some more observations:
1. When I plug-in the network cable to the LAN linux pc, tcpdump on this pc shows the following:
66:3e:1c:xx:xx:xx > a9:36:97:xx:xx:xx, ethertype Unknown (0x6034), length 1536:
        0x0000:  8203 f88c d95f 59e4 0893 b3fe bc03 6c5c  ....._Y.......l\
        0x0010:  c0b8 4c9e fbf8 eebd d769 0cf7 7c1d fcc4  ..L......i..|...
        0x0020:  2a19 8d2f a3e2 4845 3cf9 afea 7339 7444  *../..HE<...s9tD
        0x0030:  cdb1 0724 ba71 cfc6 1430 4494 872b a0ff  ...$.q...0D..+..
        0x0040:  44b1 97ef de87 6239 8f30 283e 6f87 8bf8  D.....b9.0(>o...
        0x0050:  83e6                                     ..
87:33:b7:xx:xx:xx > e2:0f:9a:xx:xx:xx, ethertype Unknown (0x5edc), length 502:
        0x0000:  9571 62e0 fe6b 8439 47ca 11af e12e 605c  .qb..k.9G.....`\
        0x0010:  26cf 7d40 8420 9cc8 74c6 5107 43e2 d11d  &.}@....t.Q.C...
        0x0020:  1cb4 0e67 06f7 9891 7580 4146 47d4 1c7b  ...g....u.AFG..{
        0x0030:  40a2 5b04 8b71 6bdc a13c 4c01 fd27 8abd  @.[..qk..<L..'..
        0x0040:  8208 2114 cb79 84d1 05c3 4eb5 b9e0 4bf1  ..!..y....N...K.
        0x0050:  98f9                                     ..
6e:a3:3b:xx:xx:xx > f3:25:80:xx:xx:xx, ethertype Unknown (0x80fe), length 1536:
        0x0000:  13c5 1e83 0821 14cb 0973 7617 fc6d a4cd  .....!...sv..m..
        0x0010:  0fbe f41e 383e c379 f880 6fde 0811 ae8c  ....8>.y..o.....
        0x0020:  13fd cac4 8e30 e7eb c3bb 6b98 0f18 1f36  .....0....k....6
        0x0030:  c79e 4184 209c c88c 7b44 f401 6995 6c7e  ..A.....{D..i.l~
        0x0040:  d042 f871 708f 195f 1a9c 23bf 7b54 1c7a  .B.qp.._..#.{T.z
        0x0050:  bec4                                     ..
8e:e4:22:xx:xx:xx > d1:af:4c:xx:xx:xx, ethertype Unknown (0xfd05), length 502:
        0x0000:  b0eb 84f3 cd7b afd3 9833 ee75 f013 1e5d  .....{...3.u...]
        0x0010:  34be cc15 18d4 c483 076b 9021 8798 a838  4........k.!...8
        0x0020:  f4fc 1aa0 2258 8c5b ebce cb33 2aa0 ff44  ...."X.[...3*..D
        0x0030:  b197 efde f7d8 cd1c 61a1 f8c0 46b5 f939  ........a...F..9
        0x0040:  6846 fd7c 0987 0d9f 2b18 37ef ecca 78d1  hF.|....+.7...x.
        0x0050:  af4c                                     .L
and in the syslog I see the messages about corrupt packet reception.

2. On the router PC I've got the following log entries from "--state INVALID":
IN=eth1 OUT= MAC=00:17:ee:7c:3f:fb:00:12:d9:54:44:73:08:00 SRC=216.239.59.103 DST=213.164.99.76 LEN=40 TOS=0x00 PREC=0x00 TTL=244 ID=52756 PROTO=TCP SPT=80 DPT=1136 WINDOW=9300 RES=0x00 RST URGP=0
IN=eth1 OUT= MAC=01:00:5e:00:00:01:00:13:d4:55:de:07:08:00 SRC=213.164.105.83 DST=224.0.0.1 LEN=36 TOS=0x00 PREC=0x00 TTL=1 ID=8847 PROTO=ICMP TYPE=9 CODE=0

3. dumps on the router with the default MTU:

external interface (internet):
$ sudo tcpdump -i eth1 -tvn icmp
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
IP (tos 0x0, ttl   1, id 8847, offset 0, flags [none], proto: ICMP (1), length: 36) 213.164.105.83 > 224.0.0.1: ICMP router advertisement lifetime 30:00 1: {213.164.105.83 0}, length 16

internal interface (LAN):
$ sudo tcpdump -i eth0 -tvn icmp
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
IP (tos 0xc0, ttl  64, id 59515, offset 0, flags [none], proto: ICMP (1), length: 576) 5.1.1.5 > 5.1.1.2: ICMP 206.82.140.163 unreachable - need to frag (mtu 576), length 556
        IP (tos 0x0, ttl 128, id 33, offset 0, flags [DF], proto: TCP (6), length: 1072) 5.1.1.2.1031 > 206.82.140.163.80: P 668629663:668630695(1032) ack 4240855996 win 65535[|icmp]


> 
> I'm not sure which of your dumps is correct and shows the problem,
> using a MTU of 400 is expected not to work.
> 
> One more thing that might be helpful is if you try to LOG all
> -m state --state INVALID packets, there is a bug related to
> ICMP NAT that might be related (we're still working on the fix).
> 
Comment 33 Alan 2008-09-26 05:53:21 UTC
Closing out stale bugs. If this bug is still present with a recent kernel please re-open the bug and update the version.

Note You need to log in before you can comment on or make changes to this bug.