Cisco <=> Cisco | Tunnel UP, but no traffic pass until "reset vpn ipsec peer" on vyos side

General questions
1

Hello
I am testing IPSEC VTI Tunnel communication between the Cisco router and Vyos VM:

Configuration Cisco

crypto ikev2 proposal vyos-ikev2-proposal
encryption aes-gcm-256
prf sha256
group 14
!
crypto ikev2 policy vyos-ikev2-policy
match address local 192.168.251.3
proposal vyos-ikev2-proposal
!
crypto ikev2 profile vyos-ikev2-profile
match identity remote address 192.168.251.1 255.255.255.255
authentication local pre-share key vyostocisco
authentication remote pre-share key vyostocisco
dpd 30 15 periodic
!
crypto ipsec transform-set vyos_transform esp-gcm 256
mode tunnel
!
!
crypto ipsec profile vyos_ipsec
set transform-set vyos_transform
set ikev2-profile vyos-ikev2-profile
!
!
interface Tunnel1000
description test VPN Tunnel to VyOS test
ip address 172.31.250.1 255.255.255.252
tunnel source 192.168.251.3
tunnel mode ipsec ipv4
tunnel destination 192.168.251.1
tunnel protection ipsec profile vyos_ipsec

VyoS configuration

set interfaces ethernet eth4 address ‘10.0.250.1/24’
set interfaces ethernet eth4 hw-id ‘50:00:00:12:00:01’
set interfaces ethernet eth5 address ‘10.113.112.101/24’
set interfaces ethernet eth5 hw-id ‘50:00:00:12:00:00’
set interfaces ethernet eth6 hw-id ‘50:00:00:12:00:02’
set interfaces ethernet eth7 hw-id ‘50:00:00:12:00:03’
set interfaces loopback lo
set interfaces vti vti0 address ‘192.168.128.237/30’
set policy route MSS-CLAMP rule 10 protocol ‘tcp’
set policy route MSS-CLAMP rule 10 set tcp-mss ‘1400’
set policy route MSS-CLAMP rule 10 tcp flags ‘SYN’
set protocols static route 0.0.0.0/0 next-hop 10.113.112.1
set protocols static route 10.232.0.0/16 next-hop 192.168.128.238
set protocols static route 192.168.120.17/32 next-hop 192.168.128.238
set service ssh port ‘22’
set system config-management commit-revisions ‘100’
set system console device ttyS0 speed ‘115200’
set system host-name ‘vyos’
set system login user vyos authentication encrypted-password ‘$6$xsFVZFpZkHuL$RbQGug1kCvfxyAPkTkAHPieqp5IXJvWIK1N7n4NGpVAqv0EjxQkwH9m0zxyZo7aoSkYwm3/1gig/4EAYL6N93.’
set system login user vyos authentication plaintext-password ‘’
set system syslog global facility daemon level ‘notice’
set system syslog global facility protocols level ‘all’
set vpn ipsec esp-group ESP_TO_CISCO compression ‘disable’
set vpn ipsec esp-group ESP_TO_CISCO lifetime ‘3600’
set vpn ipsec esp-group ESP_TO_CISCO mode ‘tunnel’
set vpn ipsec esp-group ESP_TO_CISCO pfs ‘dh-group14’
set vpn ipsec esp-group ESP_TO_CISCO proposal 1 encryption ‘aes256gcm128’
set vpn ipsec esp-group ESP_TO_CISCO proposal 1 hash ‘sha256’
set vpn ipsec ike-group IKE_TO_CISCO close-action ‘none’
set vpn ipsec ike-group IKE_TO_CISCO dead-peer-detection action ‘restart’
set vpn ipsec ike-group IKE_TO_CISCO dead-peer-detection interval ‘15’
set vpn ipsec ike-group IKE_TO_CISCO dead-peer-detection timeout ‘30’
set vpn ipsec ike-group IKE_TO_CISCO ikev2-reauth ‘yes’
set vpn ipsec ike-group IKE_TO_CISCO key-exchange ‘ikev2’
set vpn ipsec ike-group IKE_TO_CISCO lifetime ‘28800’
set vpn ipsec ike-group IKE_TO_CISCO proposal 1 dh-group ‘14’
set vpn ipsec ike-group IKE_TO_CISCO proposal 1 encryption ‘aes256gcm128’
set vpn ipsec ike-group IKE_TO_CISCO proposal 1 hash ‘sha256’
set vpn ipsec ipsec-interfaces interface ‘eth5’
set vpn ipsec site-to-site peer 192.168.138.106 authentication id ‘10.113.112.101’
set vpn ipsec site-to-site peer 192.168.138.106 authentication mode ‘pre-shared-secret’
set vpn ipsec site-to-site peer 192.168.138.106 authentication pre-shared-secret ‘vyostocisco’
set vpn ipsec site-to-site peer 192.168.138.106 authentication remote-id ‘192.168.138.106’
set vpn ipsec site-to-site peer 192.168.138.106 connection-type ‘initiate’
set vpn ipsec site-to-site peer 192.168.138.106 default-esp-group ‘ESP_TO_CISCO’
set vpn ipsec site-to-site peer 192.168.138.106 ike-group ‘IKE_TO_CISCO’
set vpn ipsec site-to-site peer 192.168.138.106 ikev2-reauth ‘inherit’
set vpn ipsec site-to-site peer 192.168.138.106 local-address ‘10.113.112.101’
set vpn ipsec site-to-site peer 192.168.138.106 vti bind ‘vti0’
set vpn ipsec site-to-site peer 192.168.138.106 vti esp-group ‘ESP_TO_CISCO’

It works ok, but after some time of no traffic through the Tunnel, no new traffic can go through the Tunnel, unless restarted on Vyos site by “reset vpn ipsec peer” command.
Is there something I did wrong in the configuration?
Please advice

Best regards

2

Hello, alabarym!
Could you please provide me with your configurations both Cisco and VyOS VM

3

It is already in my first post. Just click to unhide them

4

Outputs to clearly demonstrate the issue:
vyos@vyos:~$ show vpn debug

Status of IKE charon daemon (strongSwan 5.7.2, Linux 5.4.122-amd64-vyos, x86_64):
uptime: 65 minutes, since Jun 11 11:48:41 2021
malloc: sbrk 1994752, mmap 0, used 1160400, free 834352
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 5
loaded plugins: charon test-vectors ldap pkcs11 tpm aes rc2 sha2 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent chapoly x
cbc cmac hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default connmark stroke vici updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp
lookip error-notify certexpire led addrblock counters
Listening IP addresses:
10.113.112.101
Connections:
peer-192.168.138.106-tunnel-vti: 10.113.112.101…192.168.138.106 IKEv2, dpddelay=10s
peer-192.168.138.106-tunnel-vti: local: [10.113.112.101] uses pre-shared key authentication
peer-192.168.138.106-tunnel-vti: remote: [192.168.138.106] uses pre-shared key authentication
peer-192.168.138.106-tunnel-vti: child: 0.0.0.0/0 === 0.0.0.0/0 TUNNEL, dpdaction=restart
Security Associations (1 up, 0 connecting):
peer-192.168.138.106-tunnel-vti[2]: ESTABLISHED 60 minutes ago, 10.113.112.101[10.113.112.101]…192.168.138.106[192.168.138.106]
peer-192.168.138.106-tunnel-vti[2]: IKEv2 SPIs: 71146d09c9994c1f_i* 216134f8b13f0536_r, pre-shared key reauthentication in 6 hours
peer-192.168.138.106-tunnel-vti[2]: IKE proposal: AES_GCM_16_256/PRF_HMAC_SHA2_256/MODP_2048
peer-192.168.138.106-tunnel-vti{4}: INSTALLED, TUNNEL, reqid 2, ESP SPIs: c3b6b35f_i d9d3366d_o
peer-192.168.138.106-tunnel-vti{4}: AES_GCM_16_256/MODP_2048, 0 bytes_i, 168 bytes_o (2 pkts, 16s ago), rekeying in 27 minutes
peer-192.168.138.106-tunnel-vti{4}: 0.0.0.0/0 === 0.0.0.0/0
vyos@vyos:~ vyos@vyos:~ show interfaces vti vti0
Possible completions:
Execute the current command
brief Show summary of the specified vti interface information

vyos@vyos:~$ show interfaces vti vti0
vti0@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 10.113.112.101 peer 192.168.138.106
inet 192.168.128.237/30 scope global vti0
valid_lft forever preferred_lft forever

RX:  bytes  packets  errors  dropped  overrun       mcast
      1500       15       0        0        0           0
TX:  bytes  packets  errors  dropped  carrier  collisions
      1668       17       0        0        0           0

vyos@vyos:~$ ping 192.168.128.238
PING 192.168.128.238 (192.168.128.238) 56(84) bytes of data.
^^^C
— 192.168.128.238 ping statistics —
15 packets transmitted, 0 received, 100% packet loss, time 379ms

vyos@vyos:~$ reset vpn ipsec-peer 192.168.138.106

Resetting tunnel vti with peer 192.168.138.106…
vyos@vyos:~ ping 192.168.128.238 PING 192.168.128.238 (192.168.128.238) 56(84) bytes of data. 64 bytes from 192.168.128.238: icmp_seq=1 ttl=255 time=291 ms 64 bytes from 192.168.128.238: icmp_seq=2 ttl=255 time=291 ms 64 bytes from 192.168.128.238: icmp_seq=3 ttl=255 time=291 ms ^C --- 192.168.128.238 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 4ms rtt min/avg/max/mdev = 290.636/290.959/291.390/0.698 ms vyos@vyos:~

5

Could you please provide me with network diagram?
I believe there’s NAT somewhere in between VyOS and Cisco.
Also please send me result of “sudo swanctl -l” command at the moment where connection is established

6

I understand your confusion regarding NAT due to negotiation via UDP/4500.
Right now there an assymetry in routing. Most likely it is the case of negotiation via 4500. I will try to fix it.

Traceroute from vyos to cisco:

vyos@vyos:~$ sudo /usr/bin/traceroute -I 192.168.138.106
traceroute to 192.168.138.106 (192.168.138.106), 30 hops max, 60 byte packets
 1  10.113.112.1 (10.113.112.1)  1.223 ms  1.036 ms  1.021 ms
 2  192.168.137.194 (192.168.137.194)  0.910 ms  0.890 ms  0.794 ms
 3  192.168.132.109 (192.168.132.109)  238.676 ms  238.669 ms  238.672 ms
 4  192.168.136.170 (192.168.136.170)  291.078 ms  291.064 ms  291.057 ms
 5  192.168.138.106 (192.168.138.106)  291.049 ms  291.043 ms  290.982 ms
vyos@vyos:~$

Traceroute from cisco to vyos:

cisco#traceroute ip 10.113.112.101
Type escape sequence to abort.
Tracing the route to 10.113.112.101
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.138.105 0 msec 0 msec 0 msec
  2 192.168.136.222 70 msec 68 msec 68 msec
  3 192.168.132.62 290 msec 290 msec 290 msec
  4 192.168.140.19 292 msec 290 msec 292 msec
  5 10.113.112.101 290 msec 290 msec 292 msec

However, does the asymmetry routing appear to be a constrain to vyos IPSEC/VTI tunnels usage?

7

Please note that it also could be because Cisco might not know of dh-group14 for esp

8

Hi, @alabarym!

NAT in the middle or even different paths in two directions are not problems by themselves. However, in some cases - for example, when IPSec peers are not able to detect NAT in the middle - you may get into the situation when an IPSec connection is killed by a NAT device due to inactivity. This is pretty similar to the problem description.

Another potential issue is mentioned by @acrane DH group usage. When PFS settings for Phase 2 are not equal, a tunnel will go down after the first ESP rekeying. So, you should pay attention to this too, because in the Cisco config we do not see PFS settings for Phase 2. Thus you should check default values for this.

9

Dear @acrane

Please note that it also could be because Cisco might not know of dh-group14 for esp
Where exactly in cisco configuration I could miss not to use dh-group 14?
for example here is the list of options for cisco:

IKE
cisco(config)#crypto ikev2 proposal vyos-ikev2-proposal
cisco(config-ikev2-proposal)group ?
1 DH 768 MODP
14 DH 2048 MODP
15 DH 3072 MODP
16 DH 4096 MODP
19 DH 256 ECP
2 DH 1024 MODP
20 DH 384 ECP
21 DH 521 ECP
24 DH 2048 (256 subgroup) MODP
5 DH 1536 MODP

ESP

cisco(config)#crypto ipsec profile vyos_ipsec

cisco(ipsec-profile)#set pfs ?
  group1   D-H Group1 (768-bit modp)
  group14  D-H Group14 (2048-bit modp)
  group15  D-H Group15 (3072-bit modp)
  group16  D-H Group16 (4096-bit modp)
  group19  D-H Group19 (256-bit ecp)
  group2   D-H Group2 (1024-bit modp)
  group20  D-H Group20 (384-bit ecp)
  group21  D-H Group21 (521-bit ecp)
  group24  D-H Group24 (2048-bit modp, 256 bit subgroup)
  group5   D-H Group5 (1536-bit modp)
  <cr>
10

Hello @zsdc!

There is no really NAT in between the path.
Please find ingress/egress packer captures from the only FW in between the path.
No NAT done on the FW

11

Please also note that in pure virtual lab between Virtual Cisco CSR 1000V and Vyos VM there is not problem in communication using similar configuration on both sides.
Tunnel is UP and running with no drops all the time.

12

regarding the network diagram.
Vyos VM - Hypervisor Bridge (Eve-ng) – Layer 2 switch – Layer 2 switch - Layer 3 switch - FW – router (next encrypted tunnel via the public internet) – router – cisco router (which establish VTI/ipsec to vyos)

13

Hello @acrane @zsdc !
Could you reply to my issue?
Thank you

14

Hello, @alabarym

Sorry for delay. Could you also specify is there some routers or firewall between Cisco and VyOS in lab?

15

Hello @acrane
Yes, there are several intermediary network devices:
Vyos VM – Hypervisor Bridge (Eve-ng) – Layer 2 switch – Layer 2 switch - Layer 3 switch - Cisco ASA FW – router (next encrypted tunnel via the public internet) – router – cisco router (which establish VTI/ipsec to vyos)

16

Am I understand correctly that the IPSec tunnel should work via another IPSec tunnel (the “router (next encrypted tunnel via the public internet) – router” part)?
Did you already tried to add:

cisco(config)#crypto ipsec profile vyos_ipsec
cisco(ipsec-profile)#set pfs group14

to make configs compatible?