Bug 120441

I've encountered a bug when using AF_Packet for network capture using Bro (http://bro.org). After additional investigation, I found a bug tracking this in Bro [1], and also in Suricata [2]. The problem is that these tools (among others, such as netsniff-ng) in some cases need to leverage a symmetric hash to ensure both sides of a network connection (at least IPv4 and IPv6) are processed by the same thread of a given fanout group. In the case of Bro and Suricata, this allows a single thread to analyze both sides of the traffic for potentially malicious traffic. In netsniff-ng, this allows the processes to ensure that the captured PCAP by each process has both sides of the conversation.

Besides the evidence in bro logs that I was looking at, a more direct test is with the current master of netsniff-ng [3] that allows hash-based fanout:

For example, w/ 4 workers:

sudo nohup /usr/local/sbin/netsniff-ng -T 0xa1b2c3d4 --fanout-group 13 --fanout-type hash --mmap --ring-size 256MiB --bind-cpu 0 --silent --in eth1 --out /data/pcap/ --prefix "eth1-0." --interval 60sec  & 
sudo nohup /usr/local/sbin/netsniff-ng -T 0xa1b2c3d4 --fanout-group 13 --fanout-type hash --mmap --ring-size 256MiB --bind-cpu 1 --silent --in eth1 --out /data/pcap/ --prefix "eth1-1." --interval 60sec  & 
sudo nohup /usr/local/sbin/netsniff-ng -T 0xa1b2c3d4 --fanout-group 13 --fanout-type hash --mmap --ring-size 256MiB --bind-cpu 2 --silent --in eth1 --out /data/pcap/ --prefix "eth1-2." --interval 60sec  & 
sudo nohup /usr/local/sbin/netsniff-ng -T 0xa1b2c3d4 --fanout-group 13 --fanout-type hash --mmap --ring-size 256MiB --bind-cpu 3 --silent --in eth1 --out /data/pcap/ --prefix "eth1-3." --interval 60sec  & 


Looking at this PCAP, for TCP connections you one should see all the packets for a given 4-tuple into a given file, but this is not the case.


[1] https://bro-tracker.atlassian.net/browse/BIT-1575
[2] https://redmine.openinfosecfoundation.org/issues/1777
[3] https://github.com/netsniff-ng/netsniff-ng/