When upgrading the kernel from 2.6.29.1 to 2.6.31.4, we discovered that packets no longer get delivered to an application that is subscribing to the multicast data. We verified that multicast support was compiled into the kernel (CONFIG_IP_MULTICAST). A dump on the interface with tcpdump shows the multicast packets arriving on the network interface. However, they do not get delivered to the application. /proc/net/igmp clearly showed that we were subscribed to the relevant multicast groups, and netstat showed that the application was listening. Upgrading to 2.6.32-rc8 did not fix the problem, but downgrading to 2.6.29.1 did. This problem was seen on two different supermicro systems: one with a single (4-core) i7 940 Intel CPU, and another one with dual X5570 Xeon CPUS. In both cases the interface was on a quad-port NIC card 82571EB gigabit ethernet controller, rev 06.
(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Mon, 23 Nov 2009 22:00:45 GMT bugzilla-daemon@bugzilla.kernel.org wrote: > http://bugzilla.kernel.org/show_bug.cgi?id=14677 > > Summary: multicast client application does not receive packets > Product: Networking > Version: 2.5 > Kernel Version: 2.6.31.4-2.6.32-rc8 > Platform: All > OS/Version: Linux > Tree: Mainline > Status: NEW > Severity: normal > Priority: P1 > Component: IPV4 > AssignedTo: shemminger@linux-foundation.org > ReportedBy: berndp@optonline.net > Regression: Yes > > > When upgrading the kernel from 2.6.29.1 to 2.6.31.4, we discovered that > packets > no longer get delivered to an application that is subscribing to the > multicast > data. > > We verified that multicast support was compiled into the kernel > (CONFIG_IP_MULTICAST). A dump on the interface with tcpdump shows the > multicast > packets arriving on the network interface. However, they do not get delivered > to the application. > > /proc/net/igmp clearly showed that we were subscribed to the relevant > multicast > groups, and netstat showed that the application was listening. > > Upgrading to 2.6.32-rc8 did not fix the problem, but downgrading to 2.6.29.1 > did. > > This problem was seen on two different supermicro systems: one with a single > (4-core) i7 940 Intel CPU, and another one with dual X5570 Xeon CPUS. In both > cases the interface was on a quad-port NIC card 82571EB gigabit ethernet > controller, rev 06.
How come theres been no update on this. We see the exact same behavior, but it has to do with multiple NIC interfaces. We only get multicast traffic on the primary/first interface, not the second interface. Same thing, we see traffic on the switch port and at host via tcpdump, but multicast traffic never makes it to the application. This still isn't fixed as of 2.6.32.
Mike Coleman from Tradebot sent me the below information. I'm too busy right now to install the latest kernel and verify the workaround, but it sounds like they had the same issue at Tradebot (see Tyler Lewick's comment above), and tracked it down to an rp_filter issue. Not sure what to do with the bug report until I have verified myself that the work around is successful. I'll be installing new hardware a couple of months from now, which will be a good time to test. Here comes Mike's email: "It turns out that this commit subtly changed the behavior of rp_filter, which had the effect of causing multicast traffic to be dropped in additional cases. The basic solution is to either disable rp_filter or to add the required source routes." Here some links that Mike had in his email http://lists.netfilter.org/pipermail/netfilter/2000-September/005400.html http://kerneltrap.org/mailarchive/linux-net/2010/1/29/6737933 http://en.wikipedia.org/wiki/Reverse_path_filtering 27fed4175acf81ddd91d9a4ee2fd298981f60295 is the first bad commit commit 27fed4175acf81ddd91d9a4ee2fd298981f60295 Author: Stephen Hemminger <shemminger@vyatta.com> Date: Mon Jul 27 18:39:45 2009 -0700 ip: fix logic of reverse path filter sysctl Even though reverse path filter was changed from simple boolean to trinary control, the loose mode only works if both all and device are configured because of this logic error.
Verified that the problem is still there in 2.6.27.3 when multiple NICs are present. The work around courtesy of Mike Coleman is to switch off rp_filter. The following entries into /etc/sysctl.conf resulted in our application receiving multicast data: net.ipv4.conf.default.rp_filter = 0 net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.eth0.rp_filter = 0 net.ipv4.conf.eth1.rp_filter = 0 net.ipv4.conf.eth2.rp_filter = 0 net.ipv4.conf.eth3.rp_filter = 0 net.ipv4.conf.eth4.rp_filter = 0 net.ipv4.conf.eth5.rp_filter = 0 While this workaround takes some amount of urgency off the issue, it still looks like a bug and should be fixed.