OSPFv3 routing between VyOS and EdgeOS

Anyone seen issues w/ VyOS exchanging OSPFv3 routes w/ Ubuiti EdgeMax device? I have a very simple OSPF area 0.0.0.0 and for the life of me i can’t seem to get the EdgeMax device to pick up routes form VyOS running 1.3 OR 1.2.2.

I have confirmed that they are neighbors, but it’s as if they are not forming the adjacency which allows them to exchange routes.

show ipv6 ospfv3 neighbor
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
192.168.1.1       1    00:00:33   Twoway/DROther         00:00:56 eth1[DROther]
192.168.3.41      1    00:00:38     Full/BDR             00:00:51 eth1[DROther]
192.168.3.42      1    00:00:38     Full/DR              00:00:51 eth1[DROther]

192.168.1.1 in the above is the EdgeOS device. While the other two (2) are FRR standalone Anycast Boxes. The anycast boxes are injecting routes into my VyOS device, but never make it to the EdgeOS box.

The EdgeOS shows:

show ipv6 ospfv3 neighbor
OSPFv3 Process (*null*)
Neighbor ID     Pri   State           Dead Time   Interface  Instance ID
192.168.3.3       1   2-Way/DROther   00:00:39    eth3       0
192.168.3.41      1   Full/Backup     00:00:38    eth3       0
192.168.3.42      1   Full/DR         00:00:38    eth3       0

and the EdgeOS only has C or Connected routes plus one (1) static …

show ipv6 route
IPv6 Routing Table
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type 2, B - BGP
Timers: Uptime
IP Route Table for VRF "default"
S      ::/0 [1/0] via ::, tun0, 13:16:33
C      ::1/128 via ::, lo, 13:17:06
C      2001:470:1f04:1d29::/64 via ::, tun0, 13:16:37
C      2001:470:8759::/64 via ::, eth1, 13:16:45
C      2001:470:8759:2::/64 via ::, eth2, 13:16:46
C      2001:470:8759:3::/64 via ::, eth3, 13:16:48
C      2001:470:8759:4::/64 via ::, eth4, 13:16:37
C      2001:470:8759:5::/64 via ::, eth5, 13:16:38
C      2001:470:8759:6::/64 via ::, eth6, 13:16:40
C      2001:470:8759:7::/64 via ::, eth7, 13:16:40
C      2001:470:8759:8::/64 via ::, eth7.80, 13:16:35
C      fe80::/64 via ::, vtun0, 13:16:32

The routing table of the VyOS device:

show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route

O>* ::/0 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
O>* 2001:470:1f04:1d29::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
O>* 2001:470:8759::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
O>* 2001:470:8759::53/128 [110/20] via fe80::250:56ff:fe84:eb16, eth1, 00:46:37
O>* 2001:470:8759:2::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
O   2001:470:8759:3::/64 [110/10] is directly connected, eth1, 00:46:37
C>* 2001:470:8759:3::/64 is directly connected, eth1, 00:46:58
O   2001:470:8759:4::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
C>* 2001:470:8759:4::/64 is directly connected, eth0, 00:46:56
O>* 2001:470:8759:5::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
O>* 2001:470:8759:6::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
O>* 2001:470:8759:7::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
O>* 2001:470:8759:8::/64 [110/20] via fe80::46d9:e7ff:fe40:bd9d, eth1, 00:46:37
C>* 2001:470:8759:10::/64 is directly connected, eth4, 00:46:57
O>* 2001:470:8759:53::53/128 [110/20] via fe80::250:56ff:fe84:eb16, eth1, 00:46:37
C * fe80::/64 is directly connected, eth5, 00:46:54
C * fe80::/64 is directly connected, eth3, 00:46:56
C * fe80::/64 is directly connected, eth0, 00:46:56
C * fe80::/64 is directly connected, eth4, 00:46:57
C>* fe80::/64 is directly connected, eth1, 00:46:58

The EdgeOS device should also have these two (2) anycast routes:

O>* 2001:470:8759::53/128 [110/20] via fe80::250:56ff:fe84:eb16, eth1, 00:00:18
O>* 2001:470:8759:53::53/128 [110/20] via fe80::250:56ff:fe84:eb16, eth1, 00:00:18

My connectivity and issue is as follows:

[EdgeOS] <---- has only one static and all its connected routes
   |
   |
   v
[ VyOS ] <---- has full set of routes including those from EdgeOS & Anycast Routes
   ^
   |
   |
   v 
[anycast FRR] <-- has full set of routes including those from EdgeOS & VyOS

On VyOS:

set protocols ospfv3 area 0.0.0.0 interface 'eth1'
set protocols ospfv3 parameters router-id '192.168.3.3'

On EdgeOS:

set protocols ospfv3 area 0.0.0.0 area-type normal
set protocols ospfv3 area 0.0.0.0 interface eth1
set protocols ospfv3 area 0.0.0.0 interface eth3
set protocols ospfv3 area 0.0.0.0 interface eth2
set protocols ospfv3 default-information originate always
set protocols ospfv3 parameters abr-type cisco
set protocols ospfv3 parameters router-id 192.168.1.1
set protocols ospfv3 redistribute connected

Thoughts? suggestions on how to troubleshoot? I see the hellos going back and forth… just not establishing the adjacency. Any help is appreciated.

@ddiguru can you try use area 0 (not 0.0.0.0)?

That’s an invalid area name on EdgeOS.

set protocols ospfv3 area 0 interface eth1
Invalid OSPFv3 area: 0

Invalid OSFPv3 area "0"

Value validation failed
Set failed

Area support for OSPFv3 is not yet implemented in FRR.
http://docs.frrouting.org/en/latest/ospf6d.html#ospf6-area
Bind interface to specified area, and start sending OSPF packets. area can be specified as 0.

Unfortunately I don’t have a test platform with EdgeRouter.

I had the same challenge on VyOS as well …

set protocols ospfv3 area 0 interface eth1
[edit]
vyos@vyos-lab-r01.ddiguru.net# commit
[ protocols ospfv3 area 0 interface eth1 ]
% Unknown command: interface eth1 area 0

[[protocols ospfv3]] failed
Commit failed

On

Version:          VyOS 1.2.2
Built by:         patrick@ddiguru.com
Built on:         Mon 15 Jul 2019 10:34 UTC
Build UUID:       93b5046c-589d-4d0a-8ed0-0c1e7f126cf2
Build Commit ID:  882b223ae9a62f-dirty

Architecture:     x86_64
Boot via:         installed image
System type:      VMware guest

Hardware vendor:  VMware, Inc.
Hardware model:   VMware Virtual Platform
Hardware S/N:     VMware-42 04 0c 93 95 1d a7 54-3e 81 e8 68 8a 55 d9 d7
Hardware UUID:    42040c93-951d-a754-3e81-e8688a55d9d7

Copyright:        VyOS maintainers and contributors

@ddiguru
I create test lab with 4 vyos devices

  1. vyos emulate-edgeos
  2. vyos
  3. anycast01
  4. anycast02

All devices with area 0.0.0.0

vyos@emul-edgeos# show | commands | match ospf
set protocols ospfv3 area 0.0.0.0 interface 'eth1'
set protocols ospfv3 parameters router-id '192.168.1.1'
[edit]
vyos@emul-edgeos# 

vyos

vyos@vyos# show | commands | match ospf
set protocols ospfv3 area 0.0.0.0 interface 'eth0'
set protocols ospfv3 parameters router-id '192.168.3.3'
[edit]
vyos@vyos# 

anycast01

vyos@anycast01# show | commands | match ospf
set protocols ospfv3 area 0.0.0.0 interface 'eth1'
set protocols ospfv3 parameters router-id '192.168.3.41'
set protocols ospfv3 redistribute connected
[edit]
vyos@anycast01# 

anycast02

vyos@anycast02# show | commands | match ospf
set protocols ospfv3 area 0.0.0.0 interface 'eth1'
set protocols ospfv3 parameters router-id '192.168.3.42'
set protocols ospfv3 redistribute connected
[edit]
vyos@anycast02# 

So from router ‘anycast’ to ‘vyos’ I send 2 routes:
2001:470:8759::53/128
2001:470:8759:53::53/128

vyos@vyos:~$ show ipv6 ospfv3 neighbor 
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
192.168.1.1       1    00:00:39     Full/DROther         00:18:02 eth0[DR]
192.168.3.41      1    00:00:39     Full/BDR             00:20:42 eth0[DR]
192.168.3.42      1    00:00:31     Full/DROther         00:19:30 eth0[DR]

vyos@vyos:~$ show ipv6 route 
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

O>* 2001:470:8759::53/128 [110/1000] via fe80::5054:ff:fe51:d6d7, eth0, 00:13:11
  *                                  via fe80::5054:ff:fedb:b0f2, eth0, 00:13:11
O>* 2001:470:8759:53::53/128 [110/1000] via fe80::5054:ff:fe51:d6d7, eth0, 00:13:11
  *                                     via fe80::5054:ff:fedb:b0f2, eth0, 00:13:11

I see the same routes on emulated router ‘edgeos’

vyos@emul-edgeos# run show ipv6 ospfv3 neighbor 
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
192.168.3.3       1    00:00:31     Full/DR              00:19:08 eth1[DROther]
192.168.3.41      1    00:00:34     Full/BDR             00:19:15 eth1[DROther]
192.168.3.42      1    00:00:35   Twoway/DROther         00:19:14 eth1[DROther]

vyos@emul-edgeos# run show ipv6 ospfv3 route 
*N E1 2001:470:8759::53/128          fe80::5054:ff:fe51:d6d7     eth1 00:15:15
                                     fe80::5054:ff:fedb:b0f2     eth1 
*N E1 2001:470:8759:53::53/128       fe80::5054:ff:fe51:d6d7     eth1 00:15:15
                                     fe80::5054:ff:fedb:b0f2     eth1 
[edit]
vyos@emul-edgeos# 

Can you tell if my scheme is correct? Maybe I’m missing something?

Wow, yes, that looks to be correct. I have a pair of anycast nodes running FRR which inject the two (2) shown IPv6 loopbacks into the routing table of the VyOS router. The VyOS router is a neighbor to both of these anycast nodes and an adjacency is created. The VyOS is also a neighbor to the EdgeOS box, but the adjacency doesn’t get established correctly OR there is a problem with the anycast routes making their way into the EdgeOS routing table, but the routes in the EdgeOS box DO make it to the VyOS routing table.

What exactly is the emul-edgeos device? is that a true Ubiquiti EdgeRouter w/ EdgeOS? or some facsimile?

I’m running:

Version:      v2.0.8
Build ID:     5247496
Build on:     11/20/19 11:24
Copyright:    2012-2019 Ubiquiti Networks, Inc.
HW model:     EdgeRouter Pro 8-Port
HW S/N:       44D9E740BD9A
Uptime:       13:27:21 up 17:34,  1 user,  load average: 0.28, 0.11, 0.03

It’s VyOS system.
Ok. Can you show on the EdgeOS summary ipv6 routes?

show ipv6 route summary

abandoned this! I’ve swapped out the EdgeOS and EdgeRouter w/ VyOS 1.3 and all is well. I believe that it has to do with incompatibility or issue with FRR. I had this working at one time and i think it broke in EdgeOS v2.0.8. They implemented or revised their routing from Quagga to FRR and i bet it has to do with some compatibility issue w/ the version of FRR in EdgeOS vs VyOS.

edgeos not using frr. the use stack from ipinfusion