Bug 203483

Summary:	"ip vrf exec" does not work in docker environment
Product:	Networking	Reporter:	Ted (artisdom)
Component:	Other	Assignee:	David Ahern (dsahern)
Status:	ASSIGNED ---
Severity:	normal
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	Linux 5.0.0-13-generic #14-Ubuntu	Subsystem:
Regression:	No	Bisected commit-id:
Attachments:	1. VRF test on Ubuntu, all good. 2. VRF test on docker, failed.

Description Ted 2019-05-03 00:31:10 UTC

Created attachment 282595 [details]
1. VRF test on Ubuntu, all good.
2. VRF test on docker, failed.

"ip vrf exec" works on Ubuntu Linux 5.0.0-13-generic,
but does not work in docker environment.


In Ubuntu:

> ~$ sudo ip vrf exec Red ping 1.0.1.1
> PING 1.0.1.1 (1.0.1.1) 56(84) bytes of data.
> 64 bytes from 1.0.1.1: icmp_seq=1 ttl=64 time=0.032 ms
> 64 bytes from 1.0.1.1: icmp_seq=2 ttl=64 time=0.133 ms
> 64 bytes from 1.0.1.1: icmp_seq=3 ttl=64 time=0.039 ms
> 64 bytes from 1.0.1.1: icmp_seq=4 ttl=64 time=0.053 ms
> ^C
> --- 1.0.1.1 ping statistics ---
> 4 packets transmitted, 4 received, 0% packet loss, time 63ms
> rtt min/avg/max/mdev = 0.032/0.064/0.133/0.040 ms





In docker:

the same test failed.

> root@TUX1:~# docker run --privileged -it ubuntu bash
> root@236d33a84862:/# ip vrf exec Red ping 1.0.1.1
> PING 1.0.1.1 (1.0.1.1) 56(84) bytes of data.
> ^C
> --- 1.0.1.1 ping statistics ---
> 6 packets transmitted, 0 received, 100% packet loss, time 5118ms

Have to use "-I Red" to specifically bind to the interface to make it work.

> root@236d33a84862:/# ip vrf exec Red ping -I Red 1.0.1.1
> ping: Warning: source address might be selected on device other than Red.
> PING 1.0.1.1 (1.0.1.1) from 1.0.1.1 Red: 56(84) bytes of data.
> 64 bytes from 1.0.1.1: icmp_seq=1 ttl=64 time=0.058 ms
> 64 bytes from 1.0.1.1: icmp_seq=2 ttl=64 time=0.034 ms
> 64 bytes from 1.0.1.1: icmp_seq=3 ttl=64 time=0.067 ms
> 64 bytes from 1.0.1.1: icmp_seq=4 ttl=64 time=0.074 ms
> ^C
> --- 1.0.1.1 ping statistics ---
> 4 packets transmitted, 4 received, 0% packet loss, time 3059ms
> rtt min/avg/max/mdev = 0.034/0.058/0.074/0.016 ms


Full test logs are in attachment.

Comment 1 Ted 2019-05-07 05:41:15 UTC

I think we have found that this is because docker is mixing v1 and v2
cgroups, which is not supported by the kernel at the moment.


"While userland may start using net_prio or net_cls at any time, once either
is used, cgroup2 matching no longer works"

https://github.com/torvalds/linux/blob/master/include/linux/cgroup-defs.h#L748

adding "cgroup_no_v1=net_prio,net_cls" to the kernel parameter to disable the v1 cgroup controller will make this issue go away.

Comment 2 David Ahern 2019-05-08 15:51:36 UTC

Thanks for the follow up. I suspected something related to cgroups or nested bpf.