VXLAN packet receiving triggers ASSERT_RTNL() assertion failure if VXLAN transport interface is in different network namespace than the VXLAN interface itself: [ 38.891092] RTNL: assertion failed at /build/distro/work/shared/linux-stable-30bb3a6af25f17c356252ac6cfbfd3ec04ae1a56/net/core/net_namespace.c (187) [ 38.892738] CPU: 0 PID: 3 Comm: ksoftirqd/0 Not tainted 4.1.10-pc64-distro.git-v1.14 #1 [ 38.893720] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [ 38.894933] ffff8800ba372280 ffff88013ab3f808 ffffffff8078a62a ffff88013fc110f8 [ 38.896059] 00000000ffffffff ffff88013ab3f838 ffffffff8068b7b5 ffff88013a7a3600 [ 38.897172] ffff8800bbbc2c00 ffff8801381262a0 ffff880138126300 ffff88013ab3f848 [ 38.898354] Call Trace: [ 38.898763] [<ffffffff8078a62a>] dump_stack+0x45/0x57 [ 38.899436] [<ffffffff8068b7b5>] __peernet2id+0xa5/0xb0 [ 38.900099] [<ffffffff8068b7d8>] peernet2id+0x18/0x30 [ 38.900756] [<ffffffffa003a2ba>] vxlan_fdb_info+0xfa/0x360 [vxlan] [ 38.901526] [<ffffffff80682e57>] ? __alloc_skb+0x97/0x1e0 [ 38.902216] [<ffffffffa003a592>] vxlan_fdb_notify+0x72/0x100 [vxlan] [ 38.902994] [<ffffffffa003b426>] vxlan_fdb_create+0x136/0x370 [vxlan] [ 38.903797] [<ffffffffa003bf95>] vxlan_snoop+0x1c5/0x1d0 [vxlan] [ 38.904551] [<ffffffffa003deba>] vxlan_rcv+0x35a/0x5c0 [vxlan] [ 38.905393] [<ffffffff8069d48f>] ? dst_alloc+0x4f/0x180 [ 38.906165] [<ffffffffa003c208>] vxlan_udp_encap_recv+0x108/0x390 [vxlan] [ 38.907111] [<ffffffffa003c100>] ? vxlan_encap_bypass.isra.37+0x160/0x160 [vxlan] [ 38.908278] [<ffffffff806ff81b>] udp_queue_rcv_skb+0x35b/0x450 [ 38.909119] [<ffffffff806fff26>] __udp4_lib_rcv+0x126/0x750 [ 38.909915] [<ffffffff80700a7a>] udp_rcv+0x1a/0x20 [ 38.910640] [<ffffffff806ce5ae>] ip_local_deliver_finish+0xae/0x230 [ 38.911529] [<ffffffff806ce91a>] ip_local_deliver+0x9a/0xb0 [ 38.912341] [<ffffffff806ce218>] ip_rcv_finish+0x88/0x370 [ 38.913277] [<ffffffff806cec0f>] ip_rcv+0x2df/0x3c0 [ 38.914000] [<ffffffff80289343>] ? load_balance+0x233/0xa20 [ 38.914807] [<ffffffff80694b93>] __netif_receive_skb_core+0x6e3/0xa20 [ 38.915714] [<ffffffff8027661c>] ? update_rq_clock.part.81+0x1c/0x40 [ 38.916613] [<ffffffff80694eed>] __netif_receive_skb+0x1d/0x70 [ 38.917461] [<ffffffff80695d42>] process_backlog+0xc2/0x170 [ 38.918291] [<ffffffff806954ba>] net_rx_action+0x20a/0x340 [ 38.919123] [<ffffffff8025702f>] __do_softirq+0xef/0x320 [ 38.919959] [<ffffffff80257285>] run_ksoftirqd+0x25/0x60 [ 38.920764] [<ffffffff80273ecf>] smpboot_thread_fn+0x12f/0x190 [ 38.921605] [<ffffffff80273da0>] ? sort_range+0x30/0x30 [ 38.922384] [<ffffffff80270bb9>] kthread+0xc9/0xe0 [ 38.923104] [<ffffffff80270af0>] ? kthread_create_on_node+0x180/0x180 [ 38.923994] [<ffffffff80790e02>] ret_from_fork+0x42/0x70 [ 38.924772] [<ffffffff80270af0>] ? kthread_create_on_node+0x180/0x180 Seems to me that peernet2id() function should not be called while receiving packets or then peernet2id() function should not use rtnl lock. The issue can be reproduced with following configuration + ping (one host is enough, real network is not needed): ip netns add ns0 ip netns exec ns0 ip link set lo up ip netns add ns1 ip netns exec ns1 ip link set lo up ip netns add ns2 ip netns exec ns2 ip link set lo up ip netns add ns3 ip netns exec ns3 ip link set lo up ip link add type veth ip link set veth0 netns ns0 ip netns exec ns0 ip link set veth0 up ip netns exec ns0 ip addr add 10.0.0.1/24 dev veth0 ip link set veth1 netns ns1 ip netns exec ns1 ip link set veth1 up ip netns exec ns1 ip addr add 10.0.0.2/24 dev veth1 ip netns exec ns0 ip link add name vxlan0 type vxlan id 1000 group 224.0.0.1 local 10.0.0.1 dev veth0 learning ip netns exec ns0 ip link set vxlan0 netns ns2 ip netns exec ns2 ip link set vxlan0 up ip netns exec ns2 ip addr add 20.0.0.1/24 dev vxlan0 ip netns exec ns1 ip link add name vxlan1 type vxlan id 1000 group 224.0.0.1 local 10.0.0.2 dev veth1 learning ip netns exec ns1 ip link set vxlan1 netns ns3 ip netns exec ns3 ip link set vxlan1 up ip netns exec ns3 ip addr add 20.0.0.2/24 dev vxlan1 ip netns exec ns2 ping 20.0.0.1
Also affects 4.1.12 [ 34.412104] RTNL: assertion failed at /home/kernel/COD/linux/net/core/net_namespace.c (187) [ 34.412120] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.1.12-040112-generic #201510262131 [ 34.412123] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140709_153950- 04/01/2014 [ 34.412126] ffff880035dd2380 ffff88013fd838f8 ffffffff817e367c 0000000000000000 [ 34.412131] 00000000ffffffff ffff88013fd83928 ffffffff816cfbe5 ffff880035526200 [ 34.412135] ffff8800b9b3fe00 ffff88013a3dc9c0 ffff88013a3dccc0 ffff88013fd83938 [ 34.412140] Call Trace: [ 34.412142] <IRQ> [<ffffffff817e367c>] dump_stack+0x45/0x57 [ 34.412159] [<ffffffff816cfbe5>] __peernet2id+0xa5/0xb0 [ 34.412163] [<ffffffff816cfc08>] peernet2id+0x18/0x30 [ 34.412168] [<ffffffffc034587a>] vxlan_fdb_info+0xfa/0x360 [vxlan] [ 34.412173] [<ffffffff816c788c>] ? __alloc_skb+0x8c/0x1f0 [ 34.412178] [<ffffffffc0345b52>] vxlan_fdb_notify+0x72/0x100 [vxlan] [ 34.412182] [<ffffffffc034634a>] vxlan_fdb_create+0x14a/0x3a0 [vxlan] [ 34.412187] [<ffffffffc0346f0b>] vxlan_snoop+0x1cb/0x1d0 [vxlan] [ 34.412192] [<ffffffffc03475a2>] vxlan_rcv+0x3a2/0x620 [vxlan] [ 34.412196] [<ffffffffc03490d6>] vxlan_udp_encap_recv+0x156/0x420 [vxlan] [ 34.412201] [<ffffffffc0348f80>] ? vxlan_open+0x1e0/0x1e0 [vxlan] [ 34.412207] [<ffffffff817487b3>] udp_queue_rcv_skb+0x1f3/0x4f0 [ 34.412211] [<ffffffff817490a7>] __udp4_lib_rcv+0x127/0x930 [ 34.412215] [<ffffffff81749dda>] udp_rcv+0x1a/0x20 [ 34.412227] [<ffffffff81715e1e>] ip_local_deliver_finish+0xae/0x230 [ 34.412231] [<ffffffff81716136>] ip_local_deliver+0x46/0xa0 [ 34.412235] [<ffffffff81715a81>] ip_rcv_finish+0x81/0x370 [ 34.412239] [<ffffffff81716432>] ip_rcv+0x2a2/0x3c0 [ 34.412244] [<ffffffff816d98f3>] __netif_receive_skb_core+0x7e3/0xa10 [ 34.412248] [<ffffffff816d9b38>] __netif_receive_skb+0x18/0x60 [ 34.412252] [<ffffffff816da8e2>] process_backlog+0xb2/0x150 [ 34.412256] [<ffffffff816da0aa>] net_rx_action+0x1fa/0x320 [ 34.412265] [<ffffffff8107e093>] __do_softirq+0x103/0x280 [ 34.412269] [<ffffffff8107e39d>] irq_exit+0xad/0xb0 [ 34.412274] [<ffffffff817ed5b6>] smp_apic_timer_interrupt+0x46/0x60 [ 34.412279] [<ffffffff817eb63e>] apic_timer_interrupt+0x6e/0x80 [ 34.412280] <EOI> [<ffffffff8105fb56>] ? native_safe_halt+0x6/0x10 [ 34.412290] [<ffffffff810de4e3>] ? rcu_eqs_enter+0xa3/0xb0 [ 34.412297] [<ffffffff8101ec4e>] default_idle+0x1e/0xc0 [ 34.412315] [<ffffffff8101f7df>] arch_cpu_idle+0xf/0x20 [ 34.412325] [<ffffffff810bd557>] cpu_startup_entry+0x367/0x3e0 [ 34.412332] [<ffffffff810f48b2>] ? clockevents_register_device+0x112/0x1e0 [ 34.412338] [<ffffffff8104cb6e>] start_secondary+0x17e/0x1a0
This looks like to be fixed by the following commit in 4.2: commit 95f38411df055a0ecefe3a3d119d98241087d5ca Author: Nicolas Dichtel <nicolas.dichtel@6wind.com> Date: Thu May 7 11:02:51 2015 +0200 netns: use a spin_lock to protect nsid management Before this patch, nsid were protected by the rtnl lock. The goal of this patch is to be able to find a nsid without needing to hold the rtnl lock. The next patch will introduce a netlink socket option to listen to all netns that have a nsid assigned into the netns where the socket is opened. Thus, it's important to call rtnl_net_notifyid() outside the spinlock, to avoid a recursive lock (nsid are notified via rtnl). This was the main reason of the previous patch. Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com> Signed-off-by: David S. Miller <davem@davemloft.net>
Yes, this bug is not there in 4.2 anymore as the whole locking system has been rewritten.