VyOS IPSEC - no traffic after few hours

General questions
1

Hi,
I am facing an issue where the VPN is UP and working between 2 x VyOS but after few hours no traffic anymore. I configured script to send traffic every second to find out how long it takes to start the problem and it was ~6 hours. When the problem happens it is like below:

vyos@vyos-r1:~$ sh vpn ipsec status
IPsec Process Running: 11426
Security Associations (1 up, 0 connecting):
vyosr[13]: ESTABLISHED 29 minutes ago, 192.168.122.251[1.1.1.1]…2.2.2.2[2.2.2.2]

vyos@vyos-r1:~$ sh vpn ipsec sa
Connection State Uptime Bytes In/Out Packets In/Out Remote address Remote ID Proposal

vyos@vyos-r1:~$

I restart VPN then it works, but after few hours same problem again.

Any idea why this is happening?

2

What VyOS version are you using?

Please also provide the used configuration.

3

Hi, thanks for the replay, I am using below version and configuration. There are two VMs one connected at each side sending traffic. As a test I change “esp-group … lifetime…” to 60 and traffic stopped flowing after 60 seconds. Seems my configuration below is not able to renegotiate SA after the IPSEC Lifetime. If I enter a new Lifetime, for example 120 seconds, it establishes the SA then stop after 120 seconds and cannot re-establish automatically.

I need to use only 1 x Public IP, that is why simulating the Peer to Peer using the Loopback as Source/Destination, instead of the WAN.

vyos@vyos-r1:~$ sh version
Version: VyOS 1.4-rolling-202210090955

R1
set interfaces ethernet eth1 address ‘10.0.0.254/24’
set interfaces ethernet eth0 address ‘192.168.122.251/24’
set interfaces loopback lo address ‘1.1.1.1/32’
set interfaces vti vti5 address ‘10.10.10.2/30’
set protocols static route 2.2.2.2/32 next-hop 192.168.122.252
set protocols static route 20.0.0.0/24 interface vti5
!
set vpn ipsec esp-group central-rtr-esp lifetime ‘600’
set vpn ipsec esp-group central-rtr-esp mode ‘tunnel’
set vpn ipsec esp-group central-rtr-esp pfs ‘enable’
set vpn ipsec esp-group central-rtr-esp proposal 1 encryption ‘aes256’
set vpn ipsec esp-group central-rtr-esp proposal 1 hash ‘sha256’
set vpn ipsec ike-group central-rtr-ike key-exchange ‘ikev2’
set vpn ipsec ike-group central-rtr-ike lifetime ‘1800’
set vpn ipsec ike-group central-rtr-ike proposal 1 encryption ‘aes256’
set vpn ipsec ike-group central-rtr-ike proposal 1 hash ‘sha256’
set vpn ipsec interface ‘lo’
set vpn ipsec interface ‘eth0’
set vpn ipsec site-to-site peer vyosr authentication local-id ‘1.1.1.1’
set vpn ipsec site-to-site peer vyosr authentication mode ‘pre-shared-secret’
set vpn ipsec site-to-site peer vyosr authentication pre-shared-secret ‘text123456’
set vpn ipsec site-to-site peer vyosr authentication remote-id ‘2.2.2.2’
set vpn ipsec site-to-site peer vyosr ike-group ‘central-rtr-ike’
set vpn ipsec site-to-site peer vyosr local-address ‘1.1.1.1’
set vpn ipsec site-to-site peer vyosr remote-address ‘2.2.2.2’
set vpn ipsec site-to-site peer vyosr vti bind ‘vti5’
set vpn ipsec site-to-site peer vyosr vti esp-group ‘central-rtr-esp’

R2
set interfaces ethernet eth0 address ‘192.168.122.252/24’
set interfaces ethernet eth0 description ‘WAN’
set interfaces ethernet eth1 address ‘20.0.0.254/24’
set interfaces ethernet eth1 description ‘VM2subnet’
set interfaces loopback lo address ‘2.2.2.2/32’
set interfaces vti vti5 address ‘10.10.10.1/30’
set protocols static route 1.1.1.1/32 next-hop 192.168.122.251
set protocols static route 10.0.0.0/24 interface vti5
!
set vpn ipsec esp-group remote-rtr-esp lifetime ‘1800’
set vpn ipsec esp-group remote-rtr-esp mode ‘tunnel’
set vpn ipsec esp-group remote-rtr-esp pfs ‘enable’
set vpn ipsec esp-group remote-rtr-esp proposal 1 encryption ‘aes256’
set vpn ipsec esp-group remote-rtr-esp proposal 1 hash ‘sha256’
set vpn ipsec ike-group remote-rtr-ike key-exchange ‘ikev2’
set vpn ipsec ike-group remote-rtr-ike lifetime ‘1800’
set vpn ipsec ike-group remote-rtr-ike proposal 1 encryption ‘aes256’
set vpn ipsec ike-group remote-rtr-ike proposal 1 hash ‘sha256’
set vpn ipsec interface ‘lo’
set vpn ipsec interface ‘eth0’
set vpn ipsec site-to-site peer vyosr authentication local-id ‘2.2.2.2’
set vpn ipsec site-to-site peer vyosr authentication mode ‘pre-shared-secret’
set vpn ipsec site-to-site peer vyosr authentication pre-shared-secret ‘text123456’
set vpn ipsec site-to-site peer vyosr authentication remote-id ‘1.1.1.1’
set vpn ipsec site-to-site peer vyosr ike-group ‘remote-rtr-ike’
set vpn ipsec site-to-site peer vyosr local-address ‘2.2.2.2’
set vpn ipsec site-to-site peer vyosr remote-address ‘1.1.1.1’
set vpn ipsec site-to-site peer vyosr vti bind ‘vti5’
set vpn ipsec site-to-site peer vyosr vti esp-group 'central-rtr-esp

This is after the IPSEC 60 seconds lifetime:
64 bytes from 10.0.0.1: icmp_seq=600 ttl=62 time=1.05 ms
64 bytes from 10.0.0.1: icmp_seq=601 ttl=62 time=1.07 ms
64 bytes from 10.0.0.1: icmp_seq=602 ttl=62 time=1.33 ms
From 20.0.0.254 icmp_seq=603 Destination Net Unreachable

The result of debug:
vyos@vyos-r1:~$ show vpn debug

sudo ipsec statusall

Status of IKE charon daemon (strongSwan 5.9.1, Linux 5.15.72-amd64-vyos, x86_64):
uptime: 6 hours, since Oct 22 12:31:08 2022
malloc: sbrk 2961408, mmap 0, used 1299488, free 1661920
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 7
loaded plugins: charon test-vectors ldap pkcs11 tpm aesni aes rc2 sha2 sha1 md5 mgf1 rdrand random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ctr ccm gcm drbg curl attr kernel-netlink resolve socket-default connmark stroke vici updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp lookip error-notify certexpire led addrblock counters
Listening IP addresses:
192.168.122.251
Connections:
vyosr: 1.1.1.1…2.2.2.2 IKEv2, dpddelay=30s
vyosr: local: [1.1.1.1] uses pre-shared key authentication
vyosr: remote: [2.2.2.2] uses pre-shared key authentication
vyosr-vti: child: 0.0.0.0/0 ::/0 === 0.0.0.0/0 ::/0 TUNNEL, dpdaction=clear
Security Associations (1 up, 0 connecting):
vyosr[33]: ESTABLISHED 3 minutes ago, 1.1.1.1[1.1.1.1]…192.168.122.252[2.2.2.2]
vyosr[33]: IKEv2 SPIs: 76d5644bab07a70b_i a4e88854dc39df28_r*, rekeying in 54 minutes
vyosr[33]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024

sudo swanctl -L

vyosr: IKEv2, no reauthentication, rekeying every 3600s, dpd delay 30s
local: 1.1.1.1
remote: 2.2.2.2
local pre-shared key authentication:
id: 1.1.1.1
remote pre-shared key authentication:
id: 2.2.2.2
vyosr-vti: TUNNEL, rekeying every 3600s, dpd action is clear
local: 0.0.0.0/0 ::/0
remote: 0.0.0.0/0 ::/0

sudo swanctl -l

vyosr: #33, ESTABLISHED, IKEv2, 76d5644bab07a70b_i a4e88854dc39df28_r*
local ‘1.1.1.1’ @ 1.1.1.1[4500]
remote ‘2.2.2.2’ @ 192.168.122.252[4500]
AES_CBC-256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024
established 203s ago, rekeying in 3246s

sudo swanctl -P

sudo ip x sa show

sudo ip x policy show

src 0.0.0.0/0 dst 0.0.0.0/0
socket in priority 0 ptype main
src 0.0.0.0/0 dst 0.0.0.0/0
socket out priority 0 ptype main
src 0.0.0.0/0 dst 0.0.0.0/0
socket in priority 0 ptype main
src 0.0.0.0/0 dst 0.0.0.0/0
socket out priority 0 ptype main
src ::/0 dst ::/0
socket in priority 0 ptype main
src ::/0 dst ::/0
socket out priority 0 ptype main
src ::/0 dst ::/0
socket in priority 0 ptype main
src ::/0 dst ::/0
socket out priority 0 ptype main

sudo ip tunnel show

gre0: gre/ip remote any local any ttl inherit nopmtudisc

sudo ip rule show

0: from all lookup local
220: from all lookup 220
32766: from all lookup main
32767: from all lookup default

sudo ip route | head -100

2.2.2.2 nhid 30 via 192.168.122.252 dev eth0 proto static metric 20
10.0.0.0/24 dev eth1 proto kernel scope link src 10.0.0.254
192.168.15.0/24 dev eth2 proto kernel scope link src 192.168.15.251
192.168.122.0/24 dev eth0 proto kernel scope link src 192.168.122.251

sudo ip route show table 220

4

@alaertegv
Try the next command
When using site-to-site IPsec with VTI interfaces, be sure to disable route autoinstall
set vpn ipsec options disable-route-autoinstall

5

Thanks @a.apostoliuk. I tried using the command you shared, but still it has the result as below. I also tried searching related discussions but so far could not find the reason. I can get all packets via the IPSEC in and out without packet loss as per tests; trying to understand the number of SAs changing as below with UP an DOWNs, if anything wrong in my config or VM or VyOS image.

vyos@Claro-IPSEC# run sh vpn ipse sa |wc -l
17
[edit]
vyos@Claro-IPSEC# run sh vpn ipse sa |wc -l
18
[edit]
vyos@Claro-IPSEC# run sh vpn ipse sa |wc -l
14
[edit]

— 72.14.246.6 ping statistics —
45 packets transmitted, 45 received, 0% packet loss, time 44063ms
rtt min/avg/max/mdev = 2.350/2.885/3.517/0.256 ms
root@vm3ClaroUE:~#

vyos@Claro-IPSEC:~$ sh vpn ipsec sa
Connection State Uptime Bytes In/Out Packets In/Out Remote address Remote ID Proposal

vyosr-vti down 1s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 1s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 2s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 2s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 3s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 3s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 4s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 4s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 5s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 5s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti up 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti up 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA
2_256_128/MODP_1536
vyos@Claro-IPSEC:~$

6

@alaertegv
Try to use the next commands

set vpn ipsec site-to-site peer <PEER> connection-type 'initiate'
set vpn ipsec ike-group <IKE_GROUP_NAME> close-action 'restart'
set vpn ipsec ike-group <IKE_GROUP_NAME>  dead-peer-detection action 'restart'
set vpn ipsec ike-group <IKE_GROUP_NAME>  dead-peer-detection interval '30'
set vpn ipsec ike-group <IKE_GROUP_NAME>  dead-peer-detection timeout '120'
7

Hi @a.apostoliuk
Thanks. I tried and restart VPN. It immediatly keeps incrementing as below, difference between typing the commands is less than one 1 second. I start to believe it is my KVM environment, increase vCPUs, but still the same. I will try in other machine.

vyos@Claro-IPSEC:~$ sh vpn ipse sa
Connection State Uptime Bytes In/Out Packets In/Out Remote address Remote ID Proposal

vyosr-vti down 2s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 3s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128
vyosr-vti up 1s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyos@Claro-IPSEC:~$
vyos@Claro-IPSEC:~$
vyos@Claro-IPSEC:~$ sh vpn ipse sa
Connection State Uptime Bytes In/Out Packets In/Out Remote address Remote ID Proposal

vyosr-vti down 2s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 3s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 4s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 5s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti down 6s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyosr-vti up 1s 0B/0B 0/0 2.2.2.2 2.2.2.2 AES_CBC_256/HMAC_SHA2_256_128/MODP_1536
vyos@Claro-IPSEC:~$

8

Why do you need to listen on both eth0 and lo ?
set vpn ipsec interface ‘lo’
set vpn ipsec interface ‘eth0’
Maybe on renewing phase 2, eth0 address is being used.

The setup with eth0 and lo is weird. I would swap them to begin with.
Steps:
enable proxy arp on eth0, add /32 interface-route to DG on eth0 and add static default route to DG.

9

I’m experiencing the same thing. But traffic stops after 20-30 min. I have to restart the VPN process, then things pick back up. Has this issue been resolved on the thread?

10

Essentials clothing collection official store. Get fear of God seventh collection at
best prices. Amazing discount on all products .Find the most popular trend and patterns
in womenswear and menswear. It is amazing online store and you talk easily.