Cannot establish secured connection: Vyos <-> Cisco ASR 1001-X ipsec VTI

Bugs
, , , , ,
1

Hello
I am doing lab test in the lab between hardware Cisco ASR 1001-X and virtualized Vyatta on VMware:

Version 
vyos@vyos:~$ show version

Version: VyOS 1.3-beta-202105271929
Release Train: equuleus

Built by: autobuild@vyos.net
Built on: Thu 27 May 2021 17:51 UTC
Build UUID: 23331b23-d00e-46bd-bf08-bbdfda3fbca4
Build Commit ID: aaf3f4b58fbd3b

Architecture: x86_64
Boot via: installed image
System type: VMware guest

Hardware vendor: VMware, Inc.
Hardware model: VMware Virtual Platform
Hardware S/N: VMware-56 4d e6 b7 42 64 07 53-a4 d2 94 63 1b fb dc 24
Hardware UUID: 564de6b7-4264-0753-a4d2-94631bfbdc24

Copyright: VyOS maintainers and contributors

Vyos configuration

set interfaces ethernet eth0 address ‘192.168.251.1/24’
set interfaces vti vti0 address ‘172.31.250.2/30’
set policy route MSS-CLAMP rule 10 protocol ‘tcp’
set policy route MSS-CLAMP rule 10 set tcp-mss ‘1400’
set policy route MSS-CLAMP rule 10 tcp flags ‘SYN’
set protocols static route 0.0.0.0/0 next-hop 192.168.251.3
set protocols static route 192.168.252.0/31 next-hop 192.168.251.3
set service lldp legacy-protocols cdp
set service ssh
set system config-management commit-revisions ‘100’
set system console device ttyS0 speed ‘115200’
set system host-name ‘vyos’
set system login user vyos authentication encrypted-password ‘$6$cE.w92i.D6OhuNYr$PaFf7iNM6H3YArLy3bRxEezAP0ZgCFAyQ4EnXXZXCt./Iwq0F8l2ap67ZdQ/HpYvhPkjkCLLjzIs/6CWXZwPP.’
set system login user vyos authentication plaintext-password ‘’
set system ntp
set vpn ipsec esp-group ESP_TO_CISCO compression ‘disable’
set vpn ipsec esp-group ESP_TO_CISCO lifetime ‘3600’
set vpn ipsec esp-group ESP_TO_CISCO mode ‘tunnel’
set vpn ipsec esp-group ESP_TO_CISCO pfs ‘dh-group14’
set vpn ipsec esp-group ESP_TO_CISCO proposal 1 encryption ‘aes256gcm128’
set vpn ipsec esp-group ESP_TO_CISCO proposal 1 hash ‘sha256’
set vpn ipsec ike-group IKE_TO_CISCO close-action ‘none’
set vpn ipsec ike-group IKE_TO_CISCO dead-peer-detection action ‘restart’
set vpn ipsec ike-group IKE_TO_CISCO dead-peer-detection interval ‘15’
set vpn ipsec ike-group IKE_TO_CISCO dead-peer-detection timeout ‘30’
set vpn ipsec ike-group IKE_TO_CISCO ikev2-reauth ‘yes’
set vpn ipsec ike-group IKE_TO_CISCO key-exchange ‘ikev2’
set vpn ipsec ike-group IKE_TO_CISCO lifetime ‘28800’
set vpn ipsec ike-group IKE_TO_CISCO proposal 1 dh-group ‘14’
set vpn ipsec ike-group IKE_TO_CISCO proposal 1 encryption ‘aes256gcm128’
set vpn ipsec ike-group IKE_TO_CISCO proposal 1 hash ‘sha256’
set vpn ipsec ipsec-interfaces interface ‘eth0’
set vpn ipsec site-to-site peer 192.168.252.1 authentication id ‘192.168.251.1’
set vpn ipsec site-to-site peer 192.168.252.1 authentication mode ‘pre-shared-secret’
set vpn ipsec site-to-site peer 192.168.252.1 authentication pre-shared-secret ‘vyostocisco’
set vpn ipsec site-to-site peer 192.168.252.1 authentication remote-id ‘192.168.252.1’
set vpn ipsec site-to-site peer 192.168.252.1 connection-type ‘initiate’
set vpn ipsec site-to-site peer 192.168.252.1 default-esp-group ‘ESP_TO_CISCO’
set vpn ipsec site-to-site peer 192.168.252.1 ike-group ‘IKE_TO_CISCO’
set vpn ipsec site-to-site peer 192.168.252.1 ikev2-reauth ‘inherit’
set vpn ipsec site-to-site peer 192.168.252.1 local-address ‘192.168.251.1’
set vpn ipsec site-to-site peer 192.168.252.1 vti bind ‘vti0’
set vpn ipsec site-to-site peer 192.168.252.1 vti esp-group ‘ESP_TO_CISCO’

Outputs:

vyos@vyos:~$ show vpn debug
Status of IKE charon daemon (strongSwan 5.7.2, Linux 5.4.122-amd64-vyos, x86_64):
uptime: 32 minutes, since Jun 10 09:06:39 2021
malloc: sbrk 2015232, mmap 0, used 1351440, free 663792
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 112
loaded plugins: charon test-vectors ldap pkcs11 tpm aes rc2 sha2 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default connmark stroke vici updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp lookip error-notify certexpire led addrblock counters
Listening IP addresses:
192.168.251.1
Connections:
peer-192.168.252.1-tunnel-vti: 192.168.251.1…192.168.252.1 IKEv2, dpddelay=15s
peer-192.168.252.1-tunnel-vti: local: [192.168.251.1] uses pre-shared key authentication
peer-192.168.252.1-tunnel-vti: remote: [192.168.252.1] uses pre-shared key authentication
peer-192.168.252.1-tunnel-vti: child: 0.0.0.0/0 === 0.0.0.0/0 TUNNEL, dpdaction=restart
Security Associations (0 up, 0 connecting):
none

vyos@vyos:~$ show vpn ike sa
Peer ID / IP Local ID / IP

192.168.252.1 192.168.251.1

State  IKEVer  Encrypt  Hash    D-H Group      NAT-T  A-Time  L-Time
-----  ------  -------  ----    ---------      -----  ------  ------
down   N/A     n/a      n/a     n/a(n/a)       no     0       n/a

vyos@vyos:~$ show vpn ipsec
Possible completions:
policy Show the in-kernel crypto policies
sa Show all active IPsec Security Associations (SA)
state Show the in-kernel crypto state
status Show status of IPsec process

vyos@vyos:~$ show vpn ipsec sa
Connection State Uptime Bytes In/Out Packets In/Out Remote address Remote ID Proposal

vyos@vyos:~$

Cisco ASR 1001-X

Cisco Configuration

cisco#sh run | sec crypto
crypto ikev2 proposal vyos-ikev2-proposal
encryption aes-gcm-256
prf sha256
group 14
crypto ikev2 policy vyos-ikev2-policy
match address local 192.168.252.1
proposal vyos-ikev2-proposal
crypto ikev2 profile vyos-ikev2-profile
match identity remote address 192.168.251.1 255.255.255.255
authentication remote pre-share key vyostocisco
authentication local pre-share key vyostocisco
dpd 30 15 periodic
crypto ipsec transform-set vyos_transform esp-gcm 256
mode tunnel
crypto ipsec profile vyos_ipsec
set transform-set vyos_transform
set ikev2-profile vyos-ikev2-profile

nterface GigabitEthernet0/0/1.252
encapsulation dot1Q 252
ip address 192.168.252.1 255.255.255.254

ip route 192.168.251.0 255.255.255.0 192.168.252.0

interface Tunnel1000
description VPN Tunnel to VyOS
ip address 172.31.250.1 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source 192.168.252.1
tunnel mode ipsec ipv4
tunnel destination 192.168.251.1
tunnel protection ipsec profile vyos_ipsec
end

Cisco outputs
cisco#show crypto ikev2 sa
cisco#show crypto ikev2 stats 

                      Crypto IKEv2 SA Statistics

System Resource Limit: 0 Max IKEv2 SAs: 0 Max in nego(in/out): 40/400
Total incoming IKEv2 SA Count: 0 active: 0 negotiating: 0
Total outgoing IKEv2 SA Count: 0 active: 0 negotiating: 0
Incoming IKEv2 Requests: 5 accepted: 5 rejected: 0
Outgoing IKEv2 Requests: 136 accepted: 136 rejected: 0
Rejected IKEv2 Requests: 0 rsrc low: 0 SA limit: 0
IKEv2 packets dropped at dispatch: 0
Incoming IKEV2 Cookie Challenged Requests: 0
accepted: 0 rejected: 0 rejected no cookie: 0
Total Deleted sessions of Cert Revoked Peers: 0

cisco#show crypto ipsec sa

interface: Tunnel1000
Crypto map tag: Tunnel1000-head-0, local addr 192.168.252.1

protected vrf: (none)
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer 192.168.251.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

 local crypto endpt.: 192.168.252.1, remote crypto endpt.: 192.168.251.1
 plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/1.252
 current outbound spi: 0x0(0)
 PFS (Y/N): N, DH group: none

 inbound esp sas:

 inbound ah sas:

 inbound pcp sas:

 outbound esp sas:

 outbound ah sas:

 outbound pcp sas:

 local crypto endpt.: 192.168.252.1, remote crypto endpt.: 192.168.251.1
 plaintext mtu 1500, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0/1.252
 current outbound spi: 0x0(0)
 PFS (Y/N): N, DH group: none

 inbound esp sas:

 inbound ah sas:

 inbound pcp sas:

 outbound esp sas:

 outbound ah sas:

 outbound pcp sas:

I would like to note that similar configuration is working fine between Cisco CSR1000V and vyos same version within Eve-NG virtualized lab

2

Reset the peer and check logs.

reset vpn ipsec-peer 192.168.252.1
show log vpn ipsec
3

here are the logs
Vyos logs:
https://www.codepile.net/pile/yLnJnoLY

Cisco debug ike below:
https://www.codepile.net/pile/RqmnKlk9

4

Try to change the connection type to respond:

set vpn ipsec site-to-site peer 192.168.252.1 connection-type ‘respond’

5

It didn’t help. what else can be done?

6

I can only guess that it this bug

So cisco sending delete ALL ike_SAs

*Jun 10 11:37:09.176: IKEv2:(SA ID = 1):[IKEv2 -> IPsec] Create IPsec SA into IPsec database                          
*Jun 10 11:37:09.177: IKEv2:(SA ID = 1):[IPsec -> IKEv2] Creation of IPsec SA into IPsec database FAILED              
*Jun 10 11:37:09.188: IKEv2:(SESSION ID = 9,SA ID = 1):: Creation/Installation of IPsec SA into IPsec DB failed       
*Jun 10 11:37:09.188: IKEv2:(SESSION ID = 9,SA ID = 1):Queuing IKE SA delete request reason: unknown                  
*Jun 10 11:37:09.197: IKEv2:(SESSION ID = 9,SA ID = 1):Sending DELETE INFO message for IPsec SA [SPI: 0xFD0BAEE6]     
*Jun 10 11:37:09.197: IKEv2:(SESSION ID = 9,SA ID = 1):Building packet for encryption.                                
Payload contents:            
 DELETE                      
*Jun 10 11:37:09.197: IKEv2:(SESSION ID = 9,SA ID = 1):Checking if request will fit in peer window                    

*Jun 10 11:37:09.197: IKEv2:(SESSION ID = 9,SA ID = 1):Sending Packet [To 192.168.251.1:500/From 192.168.252.1:500/VRF i0:f0]                                                                                                                
Initiator SPI : 4A4AF79D1E549DE2 - Responder SPI : 135F1527EE0DAD91 Message id: 2                                     
IKEv2 INFORMATIONAL Exchange REQUEST                       
Payload contents:            
 ENCR                        

*Jun 10 11:37:09.198: IKEv2:(SESSION ID = 9,SA ID = 1):Check for existing IPSEC SA                                    
*Jun 10 11:37:09.198: IKEv2:(SESSION ID = 9,SA ID = 1):Delete all IKE SAs                                             
*Jun 10 11:37:09.198: IKEv2:(SESSION ID = 9,SA ID = 1):Sending DELETE INFO message for IKEv2 SA [ISPI: 0x4A4AF79D1E549DE2 RSPI: 0x135F1527EE0DAD91]

Also, cisco uses Different IKE lifetime

IKEV2 SA created; inserting SA into database. SA lifetime timer (86400 sec)

7

I seriously doubt we are htting this bug. This test setup is in the lab, however in production there are 8 other ASR 1001-X with the same IOS-XE firmware and multiple GRE/IPSEC/VTI tunnels. The difference is all the mentioned tunnels are between Cisco-Cisco equipment

8

Dear @Viacheslav
Thanks for the catch!
I have adjusted SA lifetime to fit both sides.
However, the situation still do not move anywhere from the previous point.
What else could we check/try/change?

9

Please also note that in pure virtual lab between Virtual Cisco CSR 1000V and Vyos VM there is not problem in communication using similar configuration on both sides.
Tunnel is UP and running with no drops all the time

10

Please find relevant packet capture:

11

Hi @Viacheslav
Could you please provide an update?
Thank you

12

Try to select/change to the different proposal for IKE/ESP for both sites

I don’t see any other reasons why cisco delete SA’s

*Jun 10 11:37:09.177: IKEv2:(SA ID = 1):[IPsec -> IKEv2] Creation of IPsec SA into IPsec database FAILED              
*Jun 10 11:37:09.188: IKEv2:(SESSION ID = 9,SA ID = 1):: Creation/Installation of IPsec SA into IPsec DB failed
13

your transform set doesn’t have the hash.