Created attachment 282595 [details] 1. VRF test on Ubuntu, all good. 2. VRF test on docker, failed. "ip vrf exec" works on Ubuntu Linux 5.0.0-13-generic, but does not work in docker environment. In Ubuntu: > ~$ sudo ip vrf exec Red ping 1.0.1.1 > PING 1.0.1.1 (1.0.1.1) 56(84) bytes of data. > 64 bytes from 1.0.1.1: icmp_seq=1 ttl=64 time=0.032 ms > 64 bytes from 1.0.1.1: icmp_seq=2 ttl=64 time=0.133 ms > 64 bytes from 1.0.1.1: icmp_seq=3 ttl=64 time=0.039 ms > 64 bytes from 1.0.1.1: icmp_seq=4 ttl=64 time=0.053 ms > ^C > --- 1.0.1.1 ping statistics --- > 4 packets transmitted, 4 received, 0% packet loss, time 63ms > rtt min/avg/max/mdev = 0.032/0.064/0.133/0.040 ms In docker: the same test failed. > root@TUX1:~# docker run --privileged -it ubuntu bash > root@236d33a84862:/# ip vrf exec Red ping 1.0.1.1 > PING 1.0.1.1 (1.0.1.1) 56(84) bytes of data. > ^C > --- 1.0.1.1 ping statistics --- > 6 packets transmitted, 0 received, 100% packet loss, time 5118ms Have to use "-I Red" to specifically bind to the interface to make it work. > root@236d33a84862:/# ip vrf exec Red ping -I Red 1.0.1.1 > ping: Warning: source address might be selected on device other than Red. > PING 1.0.1.1 (1.0.1.1) from 1.0.1.1 Red: 56(84) bytes of data. > 64 bytes from 1.0.1.1: icmp_seq=1 ttl=64 time=0.058 ms > 64 bytes from 1.0.1.1: icmp_seq=2 ttl=64 time=0.034 ms > 64 bytes from 1.0.1.1: icmp_seq=3 ttl=64 time=0.067 ms > 64 bytes from 1.0.1.1: icmp_seq=4 ttl=64 time=0.074 ms > ^C > --- 1.0.1.1 ping statistics --- > 4 packets transmitted, 4 received, 0% packet loss, time 3059ms > rtt min/avg/max/mdev = 0.034/0.058/0.074/0.016 ms Full test logs are in attachment.
I think we have found that this is because docker is mixing v1 and v2 cgroups, which is not supported by the kernel at the moment. "While userland may start using net_prio or net_cls at any time, once either is used, cgroup2 matching no longer works" https://github.com/torvalds/linux/blob/master/include/linux/cgroup-defs.h#L748 adding "cgroup_no_v1=net_prio,net_cls" to the kernel parameter to disable the v1 cgroup controller will make this issue go away.
Thanks for the follow up. I suspected something related to cgroups or nested bpf.