VYOS running latest 1.5 release
vyos@VYOS-03:~$ show version
Version: VyOS 1.5-rolling-202407050020
Release train: current
Release flavor: generic
Built by: [email protected]
Built on: Fri 05 Jul 2024 02:47 UTC
Build UUID: df7f936c-c39e-41dc-9c03-89109d237213
Build commit ID: 8ec205dd21f91d
Architecture: x86_64
Boot via: installed image
System type: VMware guest
Hardware vendor: VMware, Inc.
Hardware model: VMware Virtual Platform
Hardware S/N: VMware-56 4d 53 f7 71 e6 f2 1a-b1 d5 66 b7 71 2a de 8f
Hardware UUID: 564d53f7-71e6-f21a-b1d5-66b7712ade8f
Copyright: VyOS maintainers and contributors
vyos@VYOS-03:~$
type or paste code here
have the associated bgp sessions established
vyos@VYOS-03:~$ show ip bgp summary
IPv4 Unicast Summary (VRF default):
BGP router identifier 192.168.252.2, local AS number 65536 vrf-id 0
BGP table version 436
RIB entries 127, using 12 KiB of memory
Peers 9, using 181 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.168.6.1 4 65535 431 295 436 0 0 00:05:29 49 3 N/A
192.168.6.3 4 65535 447 296 436 0 0 00:05:21 49 3 N/A
192.168.9.1 4 65535 435 300 436 0 0 00:05:29 49 3 N/A
192.168.9.3 4 65535 430 299 436 0 0 00:05:21 49 3 N/A
192.168.14.5 4 65535 599 490 436 0 0 00:05:23 15 3 N/A
192.168.14.6 4 65535 368 295 436 0 0 00:05:07 15 3 N/A
192.168.14.7 4 65535 600 489 436 0 0 00:05:12 15 3 N/A
192.168.14.8 4 65535 366 295 436 0 0 00:05:07 15 3 N/A
192.168.40.1 4 65540 246 335 436 0 0 00:20:16 0 66 N/A
Total number of neighbors 9
vyos@VYOS-03:~$
type or paste code here
This is a eBGP multipath topology, with the correct expected bgp multipath routes learned from peers
vyos@VYOS-03:~$ show ip bgp
BGP table version is 436, local router ID is 192.168.252.2, vrf id 0
Default local pref 100, local AS 65536
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*= 192.168.0.0/24 192.168.14.8 50 0 65535 65527 i
*= 192.168.14.6 50 0 65535 65527 i
* 192.168.6.3 50 0 65535 65535 65527 i
* 192.168.9.3 50 0 65535 65535 65527 i
*= 192.168.14.7 50 0 65535 65527 i
*> 192.168.14.5 50 0 65535 65527 i
* 192.168.6.1 50 0 65535 65535 65527 i
* 192.168.9.1 50 0 65535 65535 65527 i
the routes however, are rejected by underlying frr
vyos@VYOS-03:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
B>r 192.168.0.0/24 [20/50] via 192.168.14.5 (recursive), weight 1, 00:06:24
r via 192.168.14.5, vti0 onlink, weight 1, 00:06:24
r via 192.168.14.6 (recursive), weight 1, 00:06:24
r via 192.168.14.6, vti1 onlink, weight 1, 00:06:24
r via 192.168.14.7 (recursive), weight 1, 00:06:24
r via 192.168.14.7, vti2 onlink, weight 1, 00:06:24
r via 192.168.14.8 (recursive), weight 1, 00:06:24
r via 192.168.14.8, vti3 onlink, weight 1, 00:06:24
vyos@VYOS-03:~$ sudo bash
root@VYOS-03:/home/vyos# vtysh
Hello, this is FRRouting (version 9.1.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
VYOS-03# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
B>r 192.168.0.0/24 [20/50] via 192.168.14.5 (recursive), weight 1, 00:10:40
r via 192.168.14.5, vti0 onlink, weight 1, 00:10:40
r via 192.168.14.6 (recursive), weight 1, 00:10:40
r via 192.168.14.6, vti1 onlink, weight 1, 00:10:40
r via 192.168.14.7 (recursive), weight 1, 00:10:40
r via 192.168.14.7, vti2 onlink, weight 1, 00:10:40
r via 192.168.14.8 (recursive), weight 1, 00:10:40
r via 192.168.14.8, vti3 onlink, weight 1, 00:10:40
Rebooting vyos temporarily resolves the issue, until the issue occurs again…when the remote tunnel side did not send IKEv2 tunnel SA delete message, and DPD had failed to clear the tunnel’s / SA’s on VYOS side…
I already have the VYOS dpd_timeout = 30 and dpd_delay = 10
vpn {
ipsec {
ike-group VYOS-03 {
dead-peer-detection {
action clear
interval 10
timeout 30
}
key-exchange ikev2
i do firmly believe this issue is caused by a combination of
and clearly frr unable to handle this situation cleanly.
I also found that before the tunnel re-establishes from remote tunnel end point, and DPD timeout had not yet fired to clear the SA’a on VYOS side, i found that manually clearing the IKE and ESP SA’s also works around the issue, that underlying strongswan DPD failed to do, clear the tunnels !
vyos@VYOS-03:~$ reset vpn ipsec site-to-site all
Then i found this → swanctl.conf :: strongSwan Documentation
Great ! IKEv1 had major security flaws and for years had already been depreciated by IETF and the industry…any self respecting IPSEC tunnel provider had already moved to IKEv2 years ago…but running into the strongswan and frr BS above !