OSPFv3 established connection but no routes

I’m having an issue similar to OSPFv3 routing between VyOS and EdgeOS - General questions - VyOS Forums, I have a fully up to date EdgeRouter X and a VyOS 1.4 (built yesterday) router connected to a site-to-site GRE/IPsec VPN. Both sides are sending hellos and then LS Updates, LS Acks, LS Requests, and DB Descriptions, but no actual routes are exchanged.

VyOS

I’ve enabled OSPF on the GRE tunnel interface, which works perfectly (I’ve removed most of the networks from the config but the 10.83 IP subnet is connected to tun0)

 area 0 {
     network 10.83.219.172/30
     network 172.28.16.0/24
 }
 interface tun0 {
     passive {
         disable
     }
 }
 parameters {
     router-id 0.0.0.0
 }
 passive-interface default

My OSPFv3 config is similar, I’ve changed it to an example prefix, and I bumped down the MTU to account for the GRE/IPsec overhead while testing.

 area 0.0.0.0 {
     range fd99:ffff:ffff:1::/64 {
     }
 }
 interface tun0 {
     area 0.0.0.0
     ifmtu 1420
 }
 parameters {
     router-id 0.0.0.0
 }

The tunnel interface has a custom link-local address, which I saw suggested in another thread, and multicast is enabled (and confirmed working via netstat -ng). The 10.170 IP addresses correspond to a dummy interface on the VyOS side and a loopback interface on the EdgeRouter since it doesn’t support dummy interfaces. They are also the IPs in my IPsec tunnel configuration.

 address 10.83.219.173/30
 address fe80::1/64
 enable-multicast
 encapsulation gre
 mtu 1420
 parameters {
     ipv6 {
         hoplimit 255
     }
 }
 remote 10.170.168.27
 source-address 10.170.168.26

EdgeRouter

OSPF configuration:

 area 0 {
     network 192.168.1.0/24
     network 10.83.219.172/30
 }
 parameters {
     router-id 0.0.0.1
 }
 passive-interface default
 passive-interface-exclude tun1

OSPFv3 configuration:

 area 0.0.0.0 {
     interface tun1
     range fdb8:ffff:ffff:1::/64 {
     }
 }
 parameters {
     abr-type cisco
     router-id 0.0.0.1
 }
 passive-interface default
 passive-interface-exclude tun1

And my tunnel interface (tun1):

 address 10.83.219.174/30
 address fe80::2/64
 encapsulation gre
 ipv6 {
     ospfv3 {
         cost 1
         dead-interval 40
         hello-interval 10
         ifmtu 1420
         priority 1
         retransmit-interval 5
         transmit-delay 1
     }
 }
 local-ip 10.170.168.27
 mtu 1420
 multicast enable
 remote-ip 10.170.168.26
 ttl 255

LSAs

If I trigger an update by restarting OSPF, this is what my OSPFv3 traffic looks like. Unfortunately, the DB descriptions only contain type 1, type 8, and type 10 LSAs, when I believe there should be type 3 LSAs for my routes (forgive my lack of OSPFv3 knowledge). There are no subnets being shared in any of those LSAs.

The OSPF routing tables (show ipv6 route ospfv3 or ospf6) on either side contain no routes after this exchange. I don’t see any errors on the EdgeRouter, and I couldn’t figure out how to enable more detailed logs for OSPFv3 on VyOS. On VyOS, my neighbors list is:

user@fw# run show ipv6 ospfv3 neighbor
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
0.0.0.1           1    00:00:38     Full/PointToPoint    00:01:57 tun0[PointToPoint]

As a side note, although I specified the router-id on VyOS to be 0.0.0.0, as soon as the daemon starts I think it sends one hello with that router ID and then changes it to the IP address of another interface on the system. Hence, the neighbors on the EdgeRouter are:

user@edge# run show ipv6 ospfv3 neighbor
OSPFv3 Process (*null*)
Neighbor ID     Pri   State           Dead Time   Interface  Instance ID
172.28.19.62      1   Full/ -         00:00:33    tun1       0

Did you check network type point-to-point?

https://docs.vyos.io/en/latest/configuration/protocols/ospf.html#cfgcmd-set-protocols-ospfv3-interface-interface-network-type

I didn’t before, no. I tried setting that now as well, to both point-to-point and broadcast, but there isn’t a corresponding option I could find on the EdgeRouter, and it didn’t appear to make a difference. Also, I enabled FRR debugging and I get this error when trying to commit with those options:

reload_configuration: Reloading config using temporary file: /tmp/tmp6286nfy5
reload_configuration: Executing command against frr-reload: "/usr/lib/frr/frr-reload.py --reload --daemon ospf6d /tmp/tmp6286nfy5"
frr-reload output:   0 line 4: % Command incomplete[42]: interface tun0
frr-reload output:   1
frr-reload output:   2 line 5: % Unknown command[42]:  ipv6 ospf6 network point-to-point

Bumping, and I’m also running into an issue with an OpenVPN TAP tunnel to the same destination. It works over OSPFv2, but I can’t even add the bridge interface to the OSPFv3 config (and since I can’t add dummy interfaces to OSPFv3, I can’t just use those in the bridge connected to the TAP interface). Here’s the config for the tunnel:

 openvpn vtun0 {
     description "Site-Home OpenVPN"
     device-type tap
     encryption {
         cipher aes256
     }
     hash sha256
     local-port 1194
     mode site-to-site
     openvpn-option "--proto udp4"
     persistent-tunnel
     remote-host home.example.com
     remote-port 1194
     shared-secret-key s2s
 }
 bridge br0 {
     address 10.11.3.205/30
     address fe80::1/64
     description "Site-Home OpenVPN endpoint"
     member {
         interface vtun0 {
         }
     }
 }

Trying to run set protocols ospfv3 interface br0 area 0.0.0.0 and then committing fails with the error:

frr-reload output: 192 2023-05-06 17:28:59,131  INFO: /var/run/frr/reload-IOTZAG.txt content
frr-reload output: 193 ['router ospf6\n ospf6 router-id 0.0.0.0\n',
frr-reload output: 194  'interface br0\n',
frr-reload output: 195  'interface br0\n ipv6 ospf6 area 0.0.0.0\n',
frr-reload output: 196  'router ospf6\n ospf6 router-id 0.0.0.0\n',
frr-reload output: 197  'interface br0\n',
frr-reload output: 198  'interface br0\n ipv6 ospf6 area 0.0.0.0\n']
frr-reload output: 199 line 4: % Command incomplete[42]: interface br0
frr-reload output: 200
frr-reload output: 201 line 6: % Command incomplete[42]: interface br0
frr-reload output: 202
frr-reload output: 203 line 7: % Unknown command[42]:  ipv6 ospf6 area 0.0.0.0
frr-reload output: 204 line 12: % Command incomplete[42]: interface br0
frr-reload output: 205
frr-reload output: 206 line 14: % Command incomplete[42]: interface br0
frr-reload output: 207
frr-reload output: 208 line 15: % Unknown command[42]:  ipv6 ospf6 area 0.0.0.0
frr-reload output: 209 2023-05-06 17:28:59,170 WARNING: frr-reload.py failed due to
frr-reload output: 210 vtysh (exec file) exited with status 2

Did you check if networks you want to advertise are present in local OSPF database?

No, they don’t appear in ipv6 ospfv3 route, ipv6 route ospfv3, or any other OSPFv3-related command. Based on the OSPFv2 commands, I’d expect to see them (and the remote routes) in both of those locations. I do, however, see the link states for both sides in the database, along with some other entries:

        Area Scoped Link State Database (Area 0.0.0.0)

Age:   76 Type: Router
Link State ID: 0.0.0.0
Advertising Router: 0.0.0.1
LS Sequence Number: 0x8000015c
CheckSum: 0x3520 Length: 40
Duration: 00:01:06
    Bits: -------- Options: --|-|AF|-|-|DC|R|-|--|E|V6
    Type: Point-To-Point Metric: 10
    Interface ID: 0.0.0.23
    Neighbor Interface ID: 0.0.0.18
    Neighbor Router ID: 172.28.19.62

Age:   76 Type: Router
Link State ID: 0.0.0.0
Advertising Router: 172.28.19.62
LS Sequence Number: 0x80000002
CheckSum: 0xa520 Length: 40
Duration: 00:01:16
    Bits: -------- Options: --|-|--|-|-|--|R|-|--|E|V6
    Type: Transit-Network Metric: 1
    Interface ID: 0.0.0.18
    Neighbor Interface ID: 0.0.0.18
    Neighbor Router ID: 172.28.19.62

Age:   76 Type: Network
Link State ID: 0.0.0.18
Advertising Router: 172.28.19.62
LS Sequence Number: 0x80000001
CheckSum: 0x15ab Length: 32
Duration: 00:01:16
     Options: --|-|AF|-|-|DC|R|-|--|E|V6
     Attached Router: 172.28.19.62
     Attached Router: 0.0.0.1

Age:   76 Type: 0xa00a
Link State ID: 0.0.0.23
Advertising Router: 0.0.0.1
LS Sequence Number: 0x80000158
CheckSum: 0x88a5 Length: 92
Duration: 00:01:06
        Unknown contents:

        00030010 00020040 00110008 00000012
        ac1c133e 00010001 01000000 00050004
        0000000a 00080020 00000000 00000000
        00000000 00000000 00000000 00000000
        00000000 00000000

Age: 1492 Type: 0xa00a
Link State ID: 0.0.0.31
Advertising Router: 0.0.0.1
LS Sequence Number: 0x80000023
CheckSum: 0x8dbd Length: 32
Duration: 00:01:16
        Unknown contents:

        00030010 00020000 00110000

        I/F Scoped Link State Database (I/F tun0 in Area 0.0.0.0)

Age: 1152 Type: Link
Link State ID: 0.0.0.23
Advertising Router: 0.0.0.1
LS Sequence Number: 0x8000013b
CheckSum: 0x63dc Length: 44
Duration: 00:00:02
     Priority: 1 Options: --|-|AF|-|-|DC|R|-|--|E|V6
     LinkLocal Address: fe80::2
     Number of Prefix: 0

Age:   42 Type: Link
Link State ID: 0.0.0.18
Advertising Router: 172.28.19.62
LS Sequence Number: 0x80000001
CheckSum: 0x4940 Length: 44
Duration: 00:00:42
     Priority: 1 Options: --|-|--|-|-|--|R|-|--|E|V6
     LinkLocal Address: fe80::1
     Number of Prefix: 0

        AS Scoped Link State Database

Bumping this again, still at a loss, nothing in pcaps or logs that screams out an issue to me. The errors trying to assign an interface seem to happen for dummy interfaces, loopback, and bridges, so I can’t even test if those work rather than attaching directly to the VPN interfaces.

it doesn’t clear what is the goal here , Do you try to uses ospfv3 ipsec + gre ? or opvpn with bridge ? Could you share full configuration ? also , if you are using gre , if possible that need set ip MSS ajust :

https://docs.vyos.io/en/latest/configuration/interfaces/tunnel.html?highlight=mss#cfgcmd-set-interfaces-tunnel-interface-ip-adjust-mss-mss-clamp-mss-to-pmtu

I’m trying to use OSPFv3 over GRE/IPsec and also over OpenVPN TAP, both to the same destination. Sorry if that was confusing, I originally started with the GRE/IPsec tunnel and added the OpenVPN TAP tunnel later, and it never worked on either so I figured it might be a problem with the OSPFv3 configuration, firewall, or hopefully something simple I’m missing.

I tried setting the MSS clamp on both sides of my GRE/IPsec tunnel to 1380 (1420 - 20 for IP header - 20 for TCP header). It didn’t change anything, there are hellos from both sides and the neighbors establish but no routes are exchanged:

Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]
0.0.0.1           1    00:00:38     Full/DROther         00:00:39 tun0[DR]

I also wouldn’t really have expected MSS clamping to make a difference, my understanding is that MSS only applies to TCP, not OSPF.

Here’s my full config. I ran it through strip-private, and censored a DNS forwarding domain, the log server FQDN, and dynamic DNS username, but the rest is verbatim from the stripped config dump:

vyos.config (56.1 KB)

Thanks for the help!

EDIT: Just to clarify, I took this dump before adding the MSS clamping options. Here’s what that looks like:

 adjust-mss 1380
[edit interfaces tunnel tun0 ipv6]

It’s been over a month since the last post, but I’m no closer to figuring out what the issue is. Are there any ways for me to troubleshoot this problem further?