OSPF issue since power outage

General questions
1

Hello,

I had a power outage and when my Vyos router went back online, the config was there except the OSPF configuration. It is strange because this OSPF config was here for years and properly saved. It did not lost anything except OSPF.

I recreated it but for some reason, I am totally unable to receive the ospf route in my table.
I have a second router with a similar config and it works fine.

Version: VyOS 1.4-rolling-202102060218
Release Train: sagitta

See below some command input including the config (anonymized)

vyos@MM-GG-BR2:~$ show ip ospf neighbor

Neighbor ID     Pri State           Dead Time Address         Interface                        RXmtL RqstL DBsmL
22.30.74.1      1 Full/DR           32.533s 22.30.74.1    eth0:22.30.74.2                    0     0     0
22.30.74.3    128 2-Way/DROther     33.601s 22.30.74.3    eth0:22.30.74.2                    0     0     0
22.30.74.4    128 Full/Backup       36.217s 22.30.74.4    eth0:22.30.74.2                    0     0     0
22.30.74.6      1 2-Way/DROther     36.217s 22.30.74.6    eth0:22.30.74.2                    0     0     0

vyos@MM-GG-BR2:~$ show ip ospf database

       OSPF Router with ID (22.30.74.2)

                Router Link States (Area 0.0.0.0)

Link ID         ADV Router      Age  Seq#       CkSum  Link count
10.174.252.2    10.174.252.2    1567 0x80000007 0x99d3 1
10.174.252.3    10.174.252.3    1566 0x8000000a 0x5df0 2
22.30.74.1    22.30.74.1     203 0x8000000d 0xdee6 2
22.30.74.2    22.30.74.2    1572 0x8000000d 0x4dcf 1
22.30.74.3    22.30.74.3    1565 0x80000008 0x1988 2
22.30.74.4    22.30.74.4      68 0x80000006 0x57c6 1
22.30.74.6    22.30.74.6    1622 0x8000000a 0xf405 4

                Net Link States (Area 0.0.0.0)

Link ID         ADV Router      Age  Seq#       CkSum
10.174.252.1    22.30.74.3    1565 0x80000006 0x3c5e

                AS External Link States

Link ID         ADV Router      Age  Seq#       CkSum  Route
0.0.0.0         22.30.74.1     993 0x80000006 0xd7a2 E2 0.0.0.0/0 [0x0]
0.0.0.0         22.30.74.2    1592 0x80000009 0xcbaa E2 0.0.0.0/0 [0x0]
0.0.0.0         22.30.74.3      69 0x80000007 0x6f11 E2 0.0.0.0/0 [0x0]
10.62.1.0       22.30.74.2    1582 0x80000007 0x4024 E2 10.62.1.0/26 [0x0]
10.62.1.0       22.30.74.3      68 0x80000006 0x7df9 E2 10.62.1.0/26 [0x0]
10.62.5.0       22.30.74.3      68 0x80000006 0x1241 E2 10.62.5.0/27 [0x0]
10.62.10.0      22.30.74.3      68 0x80000006 0x9599 E2 10.62.10.0/24 [0x0]
10.63.2.0       22.30.74.3      68 0x80000006 0x1d39 E2 10.63.2.0/27 [0x0]
10.63.3.0       22.30.74.3      68 0x80000006 0x5124 E2 10.63.3.0/26 [0x0]
10.64.1.0       22.30.74.3      68 0x80000006 0xe055 E2 10.64.1.0/24 [0x0]
10.64.5.0       22.30.74.3      68 0x80000006 0x3938 E2 10.64.5.0/26 [0x0]
10.174.252.0    10.174.252.2     136 0x80000006 0x5611 E2 10.174.252.0/29 [0x0]
10.174.252.0    22.30.74.3      68 0x80000006 0xab28 E2 10.174.252.0/29 [0x0]
10.174.254.0    22.30.74.3      68 0x80000006 0xad20 E2 10.174.254.0/30 [0x0]
22.30.74.0    10.174.252.2     136 0x80000006 0xd616 E2 22.30.74.0/27 [0x0]
22.30.74.0    22.30.74.3      68 0x80000006 0xbc84 E2 22.30.74.0/29 [0x0]
22.30.74.0    22.30.74.4      68 0x80000006 0xb689 E2 22.30.74.0/29 [0x0]
22.30.74.8    22.30.74.3      68 0x80000006 0x6ccc E2 22.30.74.8/29 [0x0]
22.30.74.32   22.30.74.4      68 0x80000006 0xe453 E2 22.30.74.32/27 [0x0]
22.30.74.64   22.30.74.4      68 0x80000006 0x34cb E2 22.30.74.64/29 [0x0]
22.30.74.72   22.30.74.4      68 0x80000006 0xfbf7 E2 22.30.74.72/30 [0x0]
172.16.100.0    22.30.74.1    1053 0x80000006 0x3835 E2 172.16.100.0/24 [0x0]
172.16.100.0    22.30.74.2    1652 0x80000007 0x5eee E2 172.16.100.0/24 [0x0]
192.168.0.0     22.30.74.3      68 0x80000006 0xb069 E2 192.168.0.0/23 [0x0]


vyos@MM-GG-BR2:~$ show ip route ospf
vyos@MM-GG-BR2:~$

vyos@MM-GG-BR2:~$ show ip ospf interface
eth0 is up
  ifindex 3, MTU 1500 bytes, BW 1000 Mbit <UP,BROADCAST,RUNNING,MULTICAST>
  Internet Address 22.30.74.2/29, Broadcast 22.30.74.7, Area 0.0.0.0
  MTU mismatch detection: enabled
  Router ID 22.30.74.2, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DROther, Priority 1
  Backup Designated Router (ID) 22.30.74.4, Interface Address 22.30.74.4
  Multicast group memberships: OSPFAllRouters
  Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5
    Hello due in 4.040s
  Neighbor Count is 4, Adjacent neighbor count is 2
  
  
  vyos@MM-GG-BR2# show protocols ospf | commands
set area 0 area-type normal
set area 0 network '22.30.74.0/29'
set default-information originate metric-type '2'
set neighbor 22.30.74.1 priority '128'
set neighbor 22.30.74.3 poll-interval '60'
set neighbor 22.30.74.3 priority '1'
set parameters abr-type 'cisco'
set parameters router-id '22.30.74.2'
set redistribute connected metric-type '2'
set redistribute static metric-type '2'
set redistribute static route-map 'OSPFFILTER'
[edit]

vyos@MM-GG-BR2# run show log all | match ospf
Mar 18 09:43:36 MM-GG-BR2 vyos-configd[677]: Received message: {"type": "node", "data": "/usr/libexec/vyos/conf_mode/protocols_ospf.py"}
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Update
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Already ASBR
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:1]: Update
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:1]: Already ASBR
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: LSA[Type5:0.0.0.0]: Not originate AS-external-LSA for default
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:0]: Update
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:1]: Update
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Update
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Already ASBR
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:3]: Update
Mar 18 09:43:37 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:3]: Already ASBR
Mar 18 09:50:48 MM-GG-BR2 vyos-configd[677]: Received message: {"type": "node", "data": "/usr/libexec/vyos/conf_mode/protocols_ospf.py"}
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Update
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Already ASBR
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:1]: Update
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:1]: Already ASBR
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: LSA[Type5:0.0.0.0]: Not originate AS-external-LSA for default
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:0]: Update
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:1]: Update
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Update
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:2]: Already ASBR
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:3]: Update
Mar 18 09:50:49 MM-GG-BR2 ospfd[1057]: ASBR[default:Status:3]: Already ASBR
2

Strange, the OSPF database seems to have routes.
Could you try to upgrade your VyOS to the latest rolling release?

3

Yes i just did and same thing…

 show version

Version:          VyOS 1.4-rolling-202203190317
Release train:    sagitta


show log all | match ospfd
Mar 19 08:13:24 localhost ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:32 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:32 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:33 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:33 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:39 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:39 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:40 MM-GG-BR2 ospfd[1140]: [YWPB2-VEAQY] ASBR[default:Status:1]: Update
Mar 19 15:13:40 MM-GG-BR2 ospfd[1140]: [YWPB2-VEAQY] ASBR[default:Status:2]: Update
Mar 19 15:13:40 MM-GG-BR2 ospfd[1140]: [KDBPM-JZJYC] ASBR[default:Status:2]: Already ASBR
Mar 19 15:13:40 MM-GG-BR2 ospfd[1140]: [YWPB2-VEAQY] ASBR[default:Status:3]: Update
Mar 19 15:13:40 MM-GG-BR2 ospfd[1140]: [KDBPM-JZJYC] ASBR[default:Status:3]: Already ASBR
Mar 19 15:13:40 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:40 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:42 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:42 MM-GG-BR2 ospfd[1140]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Mar 19 15:13:45 MM-GG-BR2 ospfd[1140]: [TBZ37-E2AXD][EC 100663310] snmp[err]: MIB search path: $HOME/.snmp/mibs:/usr/share/snmp/mibs:/usr/share/snmp/mibs/iana:/usr/share/snmp/mibs/ietf
Mar 19 15:13:45 MM-GG-BR2 ospfd[1140]: [TBZ37-E2AXD][EC 100663310] snmp[err]: Cannot find module (IANAifType-MIB): At line 13 in /usr/share/snmp/mibs/IF-MIB.txt
Mar 19 15:13:45 MM-GG-BR2 ospfd[1140]: [RZ3YY-GPH41][EC 100663310] snmp[warning]: Did not find 'IANAifType' in module #-1 (/usr/share/snmp/mibs/IF-MIB.txt)
Mar 19 15:13:45 MM-GG-BR2 ospfd[1140]: [TBZ37-E2AXD][EC 100663310] snmp[err]: Cannot find module (IANA-RTPROTO-MIB): At line 12 in /usr/share/snmp/mibs/IP-FORWARD-MIB.txt
Mar 19 15:13:45 MM-GG-BR2 ospfd[1140]: [RZ3YY-GPH41][EC 100663310] snmp[warning]: Did not find 'IANAipRouteProtocol' in module #-1 (/usr/share/snmp/mibs/IP-FORWARD-MIB.txt)
Mar 19 15:13:45 MM-GG-BR2 ospfd[1140]: [RZ3YY-GPH41][EC 100663310] snmp[warning]: Warning: Failed to connect to the agentx master agent ([NIL]):
Mar 19 15:13:47 MM-GG-BR2 ospfd[1140]: [P4PQ9-K4XFD] DR-Election[1st]: Backup 22.30.74.4
Mar 19 15:13:47 MM-GG-BR2 ospfd[1140]: [HBZ7F-65Y86] DR-Election[1st]: DR     22.30.74.1
Mar 19 15:13:47 MM-GG-BR2 ospfd[1140]: [P4PQ9-K4XFD] DR-Election[1st]: Backup 22.30.74.4
Mar 19 15:13:47 MM-GG-BR2 ospfd[1140]: [HBZ7F-65Y86] DR-Election[1st]: DR     22.30.74.1
Mar 19 15:13:47 MM-GG-BR2 ospfd[1140]: [S5PCG-77H23] Packet[DD]: Neighbor 22.30.74.1 Negotiation done (Master).
Mar 19 15:14:00 MM-GG-BR2 ospfd[1140]: [QN9FK-3DQX7] snmp[info]: NET-SNMP version 5.9 AgentX subagent connected

vyos@MM-GG-BR2:~$ show ip route ospf
vyos@MM-GG-BR2:~$
4

Hello,

I solved the problem but it still looks very odd to me. Basically I noticed that all the neighbor routers also has the same issue. I can see the routes in the OSPF database but these are not imported in the FIB. What was really strange is these routers are from different vendor: Vyos, Mikrotik and NSX.

I assumed that the culprit could be the DR (.1) and I removed completely the OSPF configuration on .1 and repasted it. Then the routes were properly imported on ALL the other routers.

.1 is the DR so maybe it makes sense that the issue caused at its level can affect the other routers. But I still don’t understand why the other routers well receive the OSPF database but do not import the routes in the FIB. In a sense, it is like they can see there was a problem with these routes but the problem was not made visible to the network admin.