add rule: #ip ru a fwmark 10000 table 10000 pref 900 and then add vrf and salve interface #ip link add vrf-1 type vrf table 10 #ip link set dev vrf-1 up #ip link set dev ens19 master vrf-1 write a program socket bind mark 10000 to match the rule setsockopt(entry->fd, SOL_SOCKET, SO_MARK, &mark, sizeof(mark)); add ip and add route in table 10000 # ip a a 5.5.5.5/24 dev ens20 # ip r a 5.5.5.0/24 dev ens20 scope link table 10000 if I set src addr, then can't add this route visit the server,success #curl --interface 5.5.5.100 but if I use the mark, my program can't visit the server kernel can't find my socket? return RST!
your setup seems misconfigured. Why is ens20 not enslaved to vrf-1? Why are you using marks for routing and not the VRF FIB rule (l3mdev - added automatically since 4.8).
thank you for your reply！ ens20 is salved to vrf-1. I make mistake, write ens20 to ens19 in my description, sorry. But my operation is ens20. ens19 and ens20 are all salved to vrf-1 I use this rule and table to solve one problem. Problem is: if vrf-1 set two default route using different dev, # ip r s vrf vrf-1 default via 4.4.4.25 dev ens19 metric 300 default via 5.5.5.25 dev ens20 metric 301 If I bind dev ens20, but if I visit some server use default route, for example: #curl --interface ens20 7.7.7.7 kernel will select 4.4.4.25 for src addr. So I want to set mark to select correct src addr and dev. ens19 and ens20 have their own rule and table In the table, they have their own default route and link route like that： [root@localhost ~]# ip ru 900: from all fwmark 0x2710 lookup 10000 900: from all fwmark 0x2711 lookup 10001 1000: from all lookup [l3mdev-table] 32765: from all lookup local 32766: from all lookup main 32767: from all lookup default [root@localhost ~]# ip r s table 10000 default via 5.5.5.25 dev ens20 5.5.5.0/24 dev ens20 scope link [root@localhost ~]# ip r s table 10001 default via 4.4.4.25 dev ens19 4.4.4.0/24 dev ens19 scope link [root@localhost ~]# I want to use personal rule and table in vrf, so I use SO_MARK and fwmark， and then I found that kernel can't find the socket(set mark) under vrf in my operation. And I found that the route # ip r a 5.5.5.0/24 dev ens20 scope link table 10000 I can't add the src option, if I add 'src 5.5.5.5', command return "Error: Invalid prefsrc address."
maybe I find the reason. In file inet_hashtables.c, function compute_score should remove exact_dif paramter, udp has removed it. https://github.com/torvalds/linux/commit/735453730a05391b4be97ad408b3bef07df13fe7
I must add one route first # ip r a local 5.5.5.5 dev ens20 scope link src 5.5.5.5 table 10000 then I can run this command # ip r a 5.5.5.0/24 dev ens20 scope link src 5.5.5.5 table 10000 and then the table add this route correctly
type local should use scope host not scope link
There are known issues with source address selection and ECMP / multiple route options as in your setup. The VRF design has a few more limitations.