Here’s my Topology:
I’m trying to ping the loopback of vmrouter (top) from R1 (bottom) – but VyOS states “Network is unreachable” and refuses to send packets to the destination – even though the network is in the routing table. The next hops are reachable.
I have eBGP running between vmrouter and Exit1, and iBGP between exit1 and R1. Each router is redistributing connected networks to BGP.
The output from my router:
vyos@R1:~$ ping 1.2.10.1
connect: Network is unreachable
vyos@R1:~$ trace 1.2.10.1
traceroute to 1.2.10.1 (1.2.10.1), 30 hops max, 60 byte packets
connect: Network is unreachable
vyos@R1:~$ sh ip route 1.2.10.1
Routing entry for 1.2.10.1/32
Known via "bgp", distance 200, metric 0, best
Last update 03:14:18 ago
192.168.203.5 (recursive), weight 1
* 1.2.1.1, via eth0, weight 1
vyos@R1:~$ ping 192.168.203.5
PING 192.168.203.5 (192.168.203.5) 56(84) bytes of data.
64 bytes from 192.168.203.5: icmp_seq=1 ttl=63 time=3.22 ms
^C
--- 192.168.203.5 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.223/3.223/3.223/0.000 ms
vyos@R1:~$ ping 1.2.1.1
PING 1.2.1.1 (1.2.1.1) 56(84) bytes of data.
64 bytes from 1.2.1.1: icmp_seq=1 ttl=64 time=1.00 ms
^C
--- 1.2.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.003/1.003/1.003/0.000 ms
vyos@R1:~$ sh int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description
--------- ---------- --- -----------
eth0 1.2.1.2/30 u/u
eth1 - u/u
eth2 - u/u
eth3 - u/u
eth4 - u/u
eth5 - u/D
eth6 - u/D
eth7 - u/D
lo 127.0.0.1/8 u/u
::1/128
vyos@R1:~$ sh ip bgp 1.2.10.1
BGP routing table entry for 1.2.10.1/32
Paths: (1 available, best #1, table default)
Not advertised to any peer
10001
192.168.203.5 from 1.2.1.1 (1.2.10.2)
Origin incomplete, metric 0, localpref 100, valid, internal, best (First path received)
Last update: Mon May 29 16:59:51 2023
vyos@R1:~$ sh ip bgp
BGP table version is 8, local router ID is 1.2.1.2, vrf id 0
Default local pref 100, local AS 10002
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 1.2.1.0/30 0.0.0.0 0 32768 ?
* i 1.2.1.1 0 100 0 ?
*>i1.2.10.1/32 192.168.203.5 0 100 0 10001 ?
*>i1.2.10.2/32 1.2.1.1 0 100 0 i
*>i192.168.0.0/24 192.168.203.5 0 100 0 10001 ?
*>i192.168.203.0/24 1.2.1.1 0 100 0 ?
Displayed 5 routes and 6 total paths
Update
The IP is now pingable after adding and deleting a static route!
Step1: added a static route for 1.2.10.1 with next hop as 192.168.203.5 (basically replicating the BGP route with a static one).
C>* 1.2.1.0/30 is directly connected, eth0, 09:13:58
S> 1.2.10.1/32 [1/0] via 192.168.203.5 (recursive), 00:00:46
* via 1.2.1.1, eth0, 00:00:46
B 1.2.10.1/32 [200/0] via 192.168.203.5 (recursive), 11:59:30
via 1.2.1.1, eth0, 11:59:30
B>* 1.2.10.2/32 [200/0] via 1.2.1.1, eth0, 09:13:57
B> 192.168.0.0/24 [200/0] via 192.168.203.5 (recursive), 11:59:30
* via 1.2.1.1, eth0, 11:59:30
B>* 192.168.203.0/24 [200/0] via 1.2.1.1, eth0, 09:13:57
vyos@R1:~$ ping 1.2.10.1
PING 1.2.10.1 (1.2.10.1) 56(84) bytes of data.
64 bytes from 1.2.10.1: icmp_seq=1 ttl=63 time=1.72 ms
64 bytes from 1.2.10.1: icmp_seq=2 ttl=63 time=1.84 ms
^C
--- 1.2.10.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.720/1.781/1.843/0.074 ms
vyos@R1:~$ sh conf comm | grep static
set protocols static route 1.2.10.1/32 next-hop 192.168.203.5 distance '1'
This proved that the problem was indeed that VyOS didn’t inject the BGP route into the global RT even though the sh ip route command said it did.
Then I tried deleting the route to see if the target becomes unreachable when I removed the route:
vyos@R1# del proto static route 1.2.10.1/32
[edit]
[OUTPUT OMITTED]
vyos@R1:~$ sh ip route 1.2.10.1
Routing entry for 1.2.10.1/32
Known via "bgp", distance 200, metric 0, best
Last update 12:02:25 ago
192.168.203.5 (recursive), weight 1
* 1.2.1.1, via eth0, weight 1
vyos@R1:~$ ping 1.2.10.1
PING 1.2.10.1 (1.2.10.1) 56(84) bytes of data.
64 bytes from 1.2.10.1: icmp_seq=1 ttl=63 time=2.08 ms
64 bytes from 1.2.10.1: icmp_seq=2 ttl=63 time=1.64 ms
^C
--- 1.2.10.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.644/1.863/2.083/0.223 ms
But this time it stayed reachable! Is this a bug? I even rechecked the route!
vyos@R1:~$ sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* 1.2.1.0/30 is directly connected, eth0, 09:17:28
B> 1.2.10.1/32 [200/0] via 192.168.203.5 (recursive), 12:03:00
* via 1.2.1.1, eth0, 12:03:00
B>* 1.2.10.2/32 [200/0] via 1.2.1.1, eth0, 09:17:27
B> 192.168.0.0/24 [200/0] via 192.168.203.5 (recursive), 12:03:00
* via 1.2.1.1, eth0, 12:03:00
B>* 192.168.203.0/24 [200/0] via 1.2.1.1, eth0, 09:17:27