Bug 199289

Summary: deadlock with neighbor proxy and multicast_querier=1 bridges
Product: Networking Reporter: Wolfgang Bumiller (wry+bzkernel)
Component: IPV6Assignee: Hideaki YOSHIFUJI (yoshfuji)
Status: RESOLVED CODE_FIX    
Severity: normal    
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: git-master, 4.15, 4.13, 4.4 Subsystem:
Regression: No Bisected commit-id:
Attachments: journal with kernel traces

Description Wolfgang Bumiller 2018-04-04 15:24:23 UTC
Created attachment 275103 [details]
journal with kernel traces

Removing a veth which is connected to a bridge with multicast_querier=1 can
cause deadlocks of the veth's other side has neighbor-proxy entries.
Trace/log attached.

It's a bit racy and seems to be easier with full containers involved on
4.4/4.13/4.15 kernels (pretty much happening 90% of the time on my test setup)
than the commands below (which takes up to a few dozens of tries on my test VM
with a git-master kernel running).

    // Setup part:
# ip link add br0 type bridge
# ip a add fccc::1/120 dev br0
# echo 1 >/sys/devices/virtual/net/br0/bridge/multicast_querier
    // Although on git-master it happened more often when starting with this
    // at 0 and setting it to 1 after setting up the other namespace

    // Do this many times:
# ip link add a0 type veth peer name b0
# ip link set a0 up master br0
# ip netns add foo
# ip link set b0 netns foo
# ip netns exec foo ip link set b0 up
# ip netns exec foo ip a add fccc::2/120 dev b0
# ip netns exec foo ip neigh add proxy fccc::3 dev b0
    // wait a bit or ping out from the namespace foo to fccc::1
# ip link del a0
    // with some "luck" this never returns

cpus deadlock one after another with a trace similar to the one attached, tested
on git master, 4.15 (current archlinux & ubuntu kernels), and 4.13 and 4.4 on
ubuntu.

Seems to me that __neigh_create() wants to write_lock_bh() the same lock
already held in neigh_ifdown()? Another way up that path AFAICT comes via
pneigh_delete() which contrary to pneigh_ifdown() only deletes one address and
specifically unlocks before calling tbl->pdestructor()/dev_put()/kfree() where
I have not triggered the deadlock (although I haven't spent as much time on
that). Although the whole kernel side of multicast/mld/neighbor proxy/...
involved here is still a bit of a labyrinth for me to navigate.
Comment 1 Wolfgang Bumiller 2018-04-05 06:35:13 UTC
Apparently this can also be triggered without multicast_querier=1, so that may just be influencing the race to trigger it more easily on my test system.

Forgot to add the original link in the first post. The initial report is in german though: https://bugzilla.proxmox.com/show_bug.cgi?id=1713
Comment 2 Wolfgang Bumiller 2018-05-09 11:32:23 UTC
Fixed with 4.17-rc2, commit 53b76cdf7e8fecec1d09e38aad2f8579882591a8