Intra-VRF routing

alex · August 18, 2021, 2:14pm

I’d like to know if it is possible to route between two VRFs.
I could setup a leaking route, but this only ‘copies’ one existing route from one VRF to the other.
I’ld like to be able to setup the default GW from one VRF to the other, so that the routing decision not resolved by the 1st VRF gets handled by the 2nd VRF.

An other solution would be, if I could bring up a 3rd VRF that establish iBGP to the other two VRFs

c-po · August 18, 2021, 2:47pm

Dynamic routing (e.g. BGP or OSPF) was added in the current development cycle which is VyOS 1.4…

See VRF — VyOS 1.4.x (sagitta) documentation for more information

alex · August 18, 2021, 9:11pm

Yes, I’m using 1.4 and read the docs.
I can setup BGP in a VRF and route thru the interfaces associated to the VRF.

I could not find a way to setup a BGP session between two VRFs running at the same VyOS.

Can you help or give example how to setup the remote IP (neighbor) from BGP1-at-VRF1 to BGP2-at-VRF2 ?

Thanks

c-po · August 19, 2021, 5:10am

Latest 1.4 version added support for IPv4 SAFI to import routes from a different VRF. Maybe @fernando can help here if this would work?

fernando · August 19, 2021, 1:20pm

Hi Alex ,

yes! there are some difference when we talk about VRF, if you need leak two differences tables (vrf to vrf ) with a vrf-aware only is possible to do this with static routes . however our last version it is able to use l3vpn-vrf and adding import/export routes from a different VRF .but I have a question with respect the bgp neighbor (next-hop) ,is it reachable for VRF2? (since VRF1)? Could you draw it? it may be clear to me and I can help you with correct settings .

alex · August 19, 2021, 5:36pm

I’m able to “copy” a static route from one VRF to the other using leaking routes.

I could not rout from one VRF to the second. As a simple simulation, I would have just a few routes at MainVRF reaching CDNs and have Full-routing at the Transit-VRF.
I would like setup default-GW at Main-VRF to be the be an IP at the Transit-VRF.

For ISP-0 I would setup default-GW to the Main-VRF, so that destinations existing in CDNs get resolved direcly, but to reach other destinations, routing should get an other hop to the Transit-VRF where all destinations should be resolved.

In respect to BGP-neighbore, was in a similar topology, with 3 VRFs
1st VRF for CDN
2nd VRF for Transit
3rd VRF-Main with iBGP to the 1st and 2nd VRF, so that the Main-VRF resolves any route

fernando · August 19, 2021, 10:49pm

I think it is possible to do something like this :

set vrf name CDN protocols bgp address-family ipv4-unicast rd vpn export '65001:1050'
set vrf name CDN protocols bgp address-family ipv4-unicast redistribute connected
set vrf name CDN protocols bgp address-family ipv4-unicast route-target vpn both '65001:1050'
set vrf name CDN protocols bgp local-as '65001'
set vrf name CDN protocols bgp neighbor 172.16.50.2 address-family ipv4-unicast
set vrf name CDN protocols bgp neighbor 172.16.50.2 ebgp-multihop '2'
set vrf name CDN protocols bgp neighbor 172.16.50.2 remote-as '65050'
set vrf name CDN protocols bgp parameters default no-ipv4-unicast
set vrf name CDN protocols static route 8.8.8.0/24 interface eth2
set vrf name CDN table '300'
set vrf name MAIN protocols bgp address-family ipv4-unicast network 10.70.70.0/24
set vrf name MAIN protocols bgp address-family ipv4-unicast rd vpn export '65001:1011'
set vrf name MAIN protocols bgp address-family ipv4-unicast redistribute connected
set vrf name MAIN protocols bgp address-family ipv4-unicast route-target vpn both '65001:1011'
set vrf name MAIN protocols bgp address-family ipv4-unicast route-target vpn import '65001:1050'
set vrf name MAIN protocols bgp address-family ipv4-unicast import vrf CDN
set vrf name MAIN protocols bgp local-as '65001'
set vrf name MAIN protocols bgp neighbor 10.70.70.2 address-family ipv4-unicast
set vrf name MAIN protocols bgp neighbor 10.70.70.2 ebgp-multihop '2'
set vrf name MAIN protocols bgp neighbor 10.70.70.2 remote-as '65035'
set vrf name MAIN protocols bgp parameters default no-ipv4-unicast

but an advanced route-policy should be made to limit the prefixes in the main vrf, this is just as an example:

vyos@MAINT# run show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

VRF CDN:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:14:24
S>* 8.8.8.0/24 [1/0] is directly connected, eth2, weight 1, 00:13:51
B>* 20.20.20.0/24 [20/0] via 172.16.50.2, eth2, weight 1, 00:03:32
C>* 172.16.50.0/24 is directly connected, eth2, 00:14:10

VRF MAIN:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:35:47
B>* 8.8.8.0/24 [200/0] is directly connected, eth2 (vrf CDN), weight 1, 00:01:11
C>* 10.70.70.0/24 is directly connected, eth0, 00:35:37
B>* 20.20.20.0/24 [200/0] via 172.16.50.2, eth2 (vrf CDN), weight 1, 00:22:27
B>* 172.16.50.0/24 [200/0] is directly connected, eth2 (vrf CDN), weight 1, 00:22:27

however we must try a little more , because I had some problems(without mpls to import prefix)

james.harr · August 23, 2021, 12:51pm

I think what Alex is talking about is what JunOS calls next-table or Cisco XR calls next-hop vrf.

It’s functionality distinct from leaking a full table and it’s functionality distinct from leaking a default route. It causes a second LPM lookup to be done after the first one matches. Only certain commercial platforms support it because some hardware can’t do multiple LPM lookups.

The specific use case I have seen is where your have a customer vrf and want to fall back to a full table for internet access. In this case, an LPM lookup is performed on the customer VRF, then if it fails (only the default route matches) it then does a second lookup in the vrf specified (the internet full table vrf)

So for this use case, the key differences leaking a full table into the customer vrf:

a full table is expensive to copy, especially if you have a large number of routes. It slows convergence, and it consumes a large amount of memory.
let’s say the customer has a /16 discard route covering their address space and then a bunch of /21 routes to actual sites. A bad /24 route for their address space that got imported from the internet table could hijack their traffic since it’s more specific than what they normally use.

Key differences from leaking a default route:

if you have 3 ISPs, receive full or even partial tables from those ISPs, then importing/leaking that default route into a customer VRF will only give them one egress path, probably not the best path either.

I don’t know how to do this in VyOS or if it’s possible, but I thought I’d attempt to add some context about a use case.

fernando · August 24, 2021, 9:57pm

Hi Guys

We’ve been worked with @c-po and we were able to add different new features on VYOS , I think it can be useful regarding INTRA-VRF routing with bgp ,let me show :

the scenario is that we have 3 vrf (CDN/MAIN/TRANSIT ) this case is just an example : MAIN can import all the prefixes , CDN can only import the prefix of a client 80.80.80.80/32 and TRANSIT export the DF-GW :

set vrf name CDN protocols bgp address-family ipv4-unicast export vpn
set vrf name CDN protocols bgp address-family ipv4-unicast import vpn
set vrf name CDN protocols bgp address-family ipv4-unicast network 172.16.50.0/24
set vrf name CDN protocols bgp address-family ipv4-unicast rd vpn export '65001:1050'
set vrf name CDN protocols bgp address-family ipv4-unicast redistribute connected
set vrf name CDN protocols bgp address-family ipv4-unicast redistribute static
set vrf name CDN protocols bgp address-family ipv4-unicast route-target vpn export '65001:1050'
set vrf name CDN protocols bgp address-family ipv4-unicast route-target vpn import '65001:1050'
set vrf name CDN protocols bgp local-as '65001'
set vrf name CDN protocols bgp neighbor 172.16.50.2 address-family ipv4-unicast
set vrf name CDN protocols bgp neighbor 172.16.50.2 ebgp-multihop '2'
set vrf name CDN protocols bgp neighbor 172.16.50.2 remote-as '65050'
set vrf name CDN protocols bgp parameters default no-ipv4-unicast
set vrf name MAIN protocols bgp address-family ipv4-unicast export vpn
set vrf name MAIN protocols bgp address-family ipv4-unicast import vpn
set vrf name MAIN protocols bgp address-family ipv4-unicast network 10.70.70.0/24
set vrf name MAIN protocols bgp address-family ipv4-unicast rd vpn export '65001:1011'
set vrf name MAIN protocols bgp address-family ipv4-unicast redistribute connected
set vrf name MAIN protocols bgp address-family ipv4-unicast route-map vpn export 'EXPORT-VPN'
set vrf name MAIN protocols bgp address-family ipv4-unicast route-target vpn export '65001:1011'
set vrf name MAIN protocols bgp address-family ipv4-unicast route-target vpn import '65001:1011 65001:1050 65001:1080'
set vrf name MAIN protocols bgp local-as '65001'
set vrf name MAIN protocols bgp neighbor 10.70.70.2 address-family ipv4-unicast
set vrf name MAIN protocols bgp neighbor 10.70.70.2 ebgp-multihop '2'
set vrf name MAIN protocols bgp neighbor 10.70.70.2 remote-as '65035'
set vrf name MAIN protocols bgp parameters default no-ipv4-unicast
set vrf name TRANSIT protocols bgp address-family ipv4-unicast export vpn
set vrf name TRANSIT protocols bgp address-family ipv4-unicast import vpn
set vrf name TRANSIT protocols bgp address-family ipv4-unicast network 192.168.80.0/24
set vrf name TRANSIT protocols bgp address-family ipv4-unicast rd vpn export '65001:1080'
set vrf name TRANSIT protocols bgp address-family ipv4-unicast redistribute connected
set vrf name TRANSIT protocols bgp address-family ipv4-unicast redistribute static
set vrf name TRANSIT protocols bgp address-family ipv4-unicast route-target vpn export '65001:1080'
set vrf name TRANSIT protocols bgp address-family ipv4-unicast route-target vpn import '65001:1080'
set vrf name TRANSIT protocols bgp local-as '65001'
set vrf name TRANSIT protocols bgp neighbor 192.168.80.2 address-family ipv4-unicast
set vrf name TRANSIT protocols bgp neighbor 192.168.80.2 ebgp-multihop '2'
set vrf name TRANSIT protocols bgp neighbor 192.168.80.2 remote-as '65040'
set vrf name TRANSIT protocols bgp parameters default no-ipv4-unicast

if we can see the tables , the MAIN vrf has all the prefixes :

[edit]      run show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

VRF CDN:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 01:40:52
S>* 8.8.8.0/24 [1/0] is directly connected, eth2, weight 1, 00:34:58
B>* 20.20.20.0/24 [20/0] via 172.16.50.2, eth2, weight 1, 01:00:12
B>* 80.80.80.0/24 [200/0] via 10.70.70.2, eth0 (vrf MAIN), weight 1, 00:16:40
C>* 172.16.50.0/24 is directly connected, eth2, 01:40:41

VRF MAIN:
B>* 0.0.0.0/0 [200/0] is directly connected, eth1 (vrf TRANSIT), weight 1, 00:19:26
K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 01:40:52
B>* 8.8.8.0/24 [200/0] is directly connected, eth2 (vrf CDN), weight 1, 00:16:42
C>* 10.70.70.0/24 is directly connected, eth0, 01:40:43
B>* 20.20.20.0/24 [200/0] via 172.16.50.2, eth2 (vrf CDN), weight 1, 00:16:42
B>* 80.80.80.0/24 [20/0] via 10.70.70.2, eth0, weight 1, 01:00:26
B   172.16.50.0/24 [200/0] is directly connected, unknown (vrf CDN) inactive, weight 1, 00:19:26
B   192.168.80.0/24 [200/0] is directly connected, unknown (vrf TRANSIT) inactive, weight 1, 00:19:26

VRF TRANSIT:
S>* 0.0.0.0/0 [1/0] is directly connected, eth1, weight 1, 00:32:30
K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 01:40:51
C>* 192.168.80.0/24 is directly connected, eth1, 00:35:46

we applied a route-map that only allows exporting theprefix 80.80.80.80/32 , but you can set any rules(community/prefix/as-path):

set policy prefix-list IPV4-EXPORT rule 10 action 'permit'
set policy prefix-list IPV4-EXPORT rule 10 le '32'
set policy prefix-list IPV4-EXPORT rule 10 prefix '80.80.80.0/24'
set policy route-map EXPORT-VPN rule 10 action 'permit'
set policy route-map EXPORT-VPN rule 10 match ip address prefix-list 'IPV4-EXPORT'
set policy route-map EXPORT-VPN rule 20 action 'deny'

we are able to communicate between MAIN/CDN

$ ping 20.20.20.20 interface 80.80.80.80
PING 20.20.20.20 (20.20.20.20) from 80.80.80.80 : 56(84) bytes of data.
64 bytes from 20.20.20.20: icmp_seq=1 ttl=63 time=5.72 ms
64 bytes from 20.20.20.20: icmp_seq=2 ttl=63 time=4.60 ms
^C
--- 20.20.20.20 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 4.597/5.158/5.720/0.561 ms