Hi, on upgrading to 3.7.x, I noticed that some of my ip rules were no longer applied. For example, I had: # ip ru sh 0: from all lookup local 32762: from 192.168.0.15 lookup foo 32763: from 1.2.3.4 lookup foo 32764: from 10.74.91.0/24 lookup foo 32765: from 192.168.0.19 lookup vpntunnel 32766: from all lookup main 32767: from all lookup default # ip ro sh table vpntunnel default via 192.168.0.144 dev br0 192.168.0.0/24 dev br0 scope link The 'lookup foo' rules were applied, but 'lookup vpntunnel' was not. I then tried to use an fwmark based rule to achieve the same result: # ip ru sh 0: from all lookup local 32761: from all fwmark 0x2 lookup vpntunnel 32762: from 192.168.0.15 lookup foo 32763: from 1.2.3.4 lookup foo 32764: from 10.74.91.0/24 lookup foo 32765: from 192.168.0.19 lookup vpntunnel 32766: from all lookup main 32767: from all lookup default Of course I made sure the appropriate packets were marked with 0x2. This worked a lot better and most packets go out via 192.168.0.144 now, but strangely, not all. I have a netfilter rule that catches the ones going the wrong way, and I get quite a few log messages along the lines of: kernel: FW: DROP: IN= OUT=eth1 SRC=192.168.0.19 DST=2.3.4.5 LEN=131 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=17504 DPT=42834 LEN=111 UID=1007 GID=1007 MARK=0x2 As you can see, the packet is logged as having the correct fwmark, but is still sent out through eth1 instead of br0. (And the IP based rule should have applied to, but didn't.) This particular result was obtained with 3.7.6-vs2.3.5.5 (linux-vserver patch), but I've seen similar behaviour with 3.7.anything. Source IP based rules worked fine with kernels up to 3.6. Confusingly, _some_ of them still work, but not all. Maybe something was broken by the removal of the route cache?
Problem still present in 3.7.10-vs2.3.5.6, fwiw. I'll try to reproduce it without the vserver patch sometime.
On a hunch I disabled the DROP rule and it turns out the packets are routed correctly. "ip ro get 2.3.4.5 from 192.168.0" reports the correct route (via 192.168.0.144 dev br0) and that's also where the kernel sends the packets. The only problem seems to be that the netfilter rule in the OUTPUT chain of the filter table is not aware of the final routing decision, which is at the very least counter-intuitive.