All routers are currently running VyOS 1.4 build 10/24 nightly
VIRTUAL NET / \ R1--1--R2 \100 100/ R3 R4 / \ MGMT NET
Above is an example of a routed network that provides high available routing for two networks. Above the horizontal line are virtual routers, and below are physical routers that serve as internet edge firewalls and connect to the management network of a small 2 system openstack cluster
for ipv4 I use simple vrrp VIP assignment, and for ipv6 I use a vrrp transition script to turn on/off router advertisements on the backup/master with a lifespan of 4 seconds and frequency max of 4
The issue that I started with that made me implement the RA change still is an issue, because traffic will die if it returns to the host via a different path that it came. I’ve observed this by running TCPdump on each in/out interface and can see it come back to an origin network router, but not forward to the origin network interface.
I do have Zone based firewall enabled, however all interfaces in this network context are in the same zone.
One factor that I also had to work around is that R1 and R2 cannot form a neighbor OSPF/OSPFv3 relationship when they are running on the same host (Routers R1-4 all communicate in area 0.0.0.0 on the same broadcast domain) . I’ve created a dedicated link between R1 and R2 with a cost of 1, and set the cost of the uplink to the management network to 100. This prevents R3 and R4 from routing traffic that should stay above the line in the virtual environment.
even with all of this, I still occasionally have connections time out because they take another path, however this has resolved the issue of connections failing 1 in 5 times to 1 in 20 times. What am I missing?
If curious, the point of this is the physical vyos routers provide internet connectivity to an openstack cluster, and I have a tenant that is used for my organization which also needs to monitor the hosts in the management network. Other tenants connect to the internet using another firewall zone.