as a seasoned Cisco engineer I’m trying to transfer some concepts to my VyOS experience.
Here’s the question - can I have GRE tunnels traffic in VRF (not in main routing table) while having tunnel interfaces in VRF main (aka GRT = General Routing Table)? Feature referred to as FVRF = Front-door VRF in Cisco lingo.
Here’s the snippet:
set interfaces ethernet eth0 address '192.168.1.150/24'
set interfaces ethernet eth0 hw-id '50:00:00:07:00:00'
set interfaces ethernet eth0 vrf 'Red'
...
set interfaces tunnel tun1 address '10.200.200.9/30'
set interfaces tunnel tun1 encapsulation 'gre'
set interfaces tunnel tun1 remote '192.168.100.100'
set interfaces tunnel tun1 source-address '192.168.1.150'
I’ve tried to implement this in the lab but no luck - tunnel is not working (connectivity between tunnel source and destination is working inside the VRF of course) Am I right that what I’m facing here is linux kernel limitations and I need to have GRE tunnel underlay network (i.e. corresponding ethernet interfaces) in main routing table, not VRF?
However I dont know if the above means that the stuff within the tunnel will be part of vrf “Blue” or if it just means that the outside of the tunnel (as in “remote” and “source-address”) will use the vrf “Blue” for routing (as in how to reach “remote”). I would guess the later.
That is you would need to setup another vrf which uses the tunnel interface as part of its nexthop.
And then configure to use that new vrf when you want to push stuff to/from the tunnel.
Something like this (not sure if next-hop is needed when you push stuff through tunnel, I would assume interface as nexthop should be enough):
set vrf name TUNNEL_GREEN description 'Tunnel_Green'
set vrf name TUNNEL_GREEN protocols static route 0.0.0.0/0 interface tun1 distance '1'
set vrf name TUNNEL_GREEN protocols static route 0.0.0.0/0 next-hop 10.200.200.8 distance '1'
set vrf name TUNNEL_GREEN table '10001'
Also note that vrf in VyOS (well actually Linux is the one to be blamed) is somewhat confusing when you come from the arista/cisco/others world.
VRF in Linux (VyOS) is just the routingtable itself - there is no isolation of the traffic as with arista/cisco/others.
For the isolation part you need to utilize NETNS (in combination with VRF) and netns is currently “work in progress” in VyOS (you can define a netns in VyOS config but you cant assig anything to it yet as it seems).
When you set vrf for interface tunnel you isolate (kinda, with the limitations of linux routing tables) tunneled traffic, not the underlay (i.e. GRE) one.
I don’t need to put tunneled traffic in separate VRF, I want to leave it in main routing table, so that’s why the question
Of course I can turn the logic upside down and leave GRE in main table and put tunnel in VRF, but I just trying to understand all the options and limitations.
I think its the other way around in VyOS (and Linux).
The VRF definition you put for “interface tunnel tun1” is which VRF (routingtable) it should be using for its underlay that is how to reach what you have defined as “remote”.
And then if you want to force lets say ntp to use the overlay of the tunnel you define something like this:
set vrf name TUNNEL_GREEN description 'Tunnel_Green'
set vrf name TUNNEL_GREEN protocols static route 0.0.0.0/0 interface tun1 distance '1'
set vrf name TUNNEL_GREEN protocols static route 0.0.0.0/0 next-hop 10.200.200.8 distance '1'
set vrf name TUNNEL_GREEN table '10001'
along with the configuration of the ntp to use the above:
If so then there doesnt seem to be many other options left.
The underlay uses main (aka vrf default) routing table. Only way to affect that is if you enable PBR or force traffic through DNAT/SNAT.
The tunnel (whats sent inside) by default uses main (vrf default) routing table but you can configure it to use vrf of your choice.
There is however “set interfaces tunnel tun1 source-interface ethX” where this could be used to use whatever vrf ethX is configured to be using.
That is you configure eth0 to use vrf blue and then you configure tun1 to use source-interface eth0. Indirectly this would mean that you have configured which vrf the underlay will be using.
I’ve tried both source address and source interface on a tunnel interface - no luck
So the only option is to user main routing table for underlay and separate vrf for runneled traffic I guess. Will do some research later (busy on other projects right now).
Am I right that what I’m facing here is linux kernel limitations and I need to have GRE tunnel underlay network (i.e. corresponding ethernet interfaces) in main routing table, not VRF?