1.2 rolling - IPSEC QM do not complete

General questions
,
1

Hi All,

I am evaluating VyOS for a project and I was testing VTI interfaces with Cisco CSR1000V.

ikev1 phase one worked without any problems.
Phase two kept failing. I am new to vyos, so I am not sure how to enable the logs, but I noticed this:

When I change the log-level using the following command:

set vpn ipsec logging log-level 1

It started working.

Then I upgraded VyOS to the latest rolling release and VPN (phase two) stopped working.

The following command made it work again:

set vpn ipsec logging log-level 2.

This is very easy to recreate: just reboot… So maybe if one can guide me with how to collect logs, I’ll be able to help myself and the project

set interfaces ethernet eth0 address '192.168.159.10/24'
set interfaces ethernet eth0 duplex 'auto'
set interfaces ethernet eth0 hw-id '00:0c:29:30:bd:0c'
set interfaces ethernet eth0 ip enable-proxy-arp
set interfaces ethernet eth0 smp-affinity 'auto'
set interfaces ethernet eth0 speed 'auto'
set interfaces ethernet eth1 address '10.118.0.100/24'
set interfaces ethernet eth1 duplex 'auto'
set interfaces ethernet eth1 hw-id '00:0c:29:30:bd:16'
set interfaces ethernet eth1 smp-affinity 'auto'
set interfaces ethernet eth1 speed 'auto'
set interfaces loopback lo address '192.168.159.100/32'
set interfaces loopback lo address '192.168.169.101/32'
set interfaces loopback lo address '192.168.159.101/32'
set interfaces vti vti0 address '10.100.0.10/24'
set interfaces vti vti0 mtu '1400'
set nat destination rule 200 destination address '192.168.159.100'
set nat destination rule 200 destination port '80'
set nat destination rule 200 inbound-interface 'eth0'
set nat destination rule 200 protocol 'tcp'
set nat destination rule 200 translation address '10.118.0.200'
set nat destination rule 200 translation port '80'
set nat source rule 10 outbound-interface 'eth0'
set nat source rule 10 source address '10.118.0.200'
set nat source rule 10 translation address '192.168.159.101'
set nat source rule 100 outbound-interface 'eth0'
set nat source rule 100 source address '10.118.0.30-10.118.0.90'
set nat source rule 100 translation address 'masquerade'
set protocols ospf area 1 network '10.118.0.0/24'
set protocols ospf area 1 network '10.100.0.0/24'
set protocols ospf passive-interface 'default'
set protocols ospf passive-interface-exclude 'vti0'
set protocols static route 0.0.0.0/0 next-hop 192.168.159.1
set protocols static route 10.119.100.1/32 next-hop 10.100.0.20
set service dhcp-server shared-network-name miniDC subnet 10.118.0.0/24 default-router '10.118.0.100'
set service dhcp-server shared-network-name miniDC subnet 10.118.0.0/24 dns-server '8.8.8.8'
set service dhcp-server shared-network-name miniDC subnet 10.118.0.0/24 range 0 start '10.118.0.30'
set service dhcp-server shared-network-name miniDC subnet 10.118.0.0/24 range 0 stop '10.118.0.90'
set service ssh port '22'
set system config-management commit-revisions '100'
set system console device ttyS0 speed '9600'
set system host-name 'vyos'
set system login user vyos authentication encrypted-password '$6$1ZgFHN6n6EmWP.X$ptiRmy3zehauj9/72XqlgqzQ3DvEbXjNjeMwDJUXaMQzkRDTGiajx9e7uIRLrr95FtpTcRYzeAPTnK9i/HSbX.'
set system login user vyos authentication plaintext-password ''
set system login user vyos level 'admin'
set system ntp server 0.pool.ntp.org
set system ntp server 1.pool.ntp.org
set system ntp server 2.pool.ntp.org
set system syslog global facility all level 'info'
set system syslog global facility protocols level 'debug'
set system time-zone 'UTC'
set vpn ipsec esp-group VTI compression 'disable'
set vpn ipsec esp-group VTI lifetime '3600'
set vpn ipsec esp-group VTI mode 'tunnel'
set vpn ipsec esp-group VTI pfs 'dh-group14'
set vpn ipsec esp-group VTI proposal 1 encryption 'aes256'
set vpn ipsec esp-group VTI proposal 1 hash 'sha1'
set vpn ipsec ike-group VTI ikev2-reauth 'no'
set vpn ipsec ike-group VTI key-exchange 'ikev1'
set vpn ipsec ike-group VTI lifetime '86400'
set vpn ipsec ike-group VTI proposal 1 dh-group '14'
set vpn ipsec ike-group VTI proposal 1 encryption 'aes256'
set vpn ipsec ike-group VTI proposal 1 hash 'sha256'
set vpn ipsec ipsec-interfaces interface 'eth0'
set vpn ipsec logging log-level '2'
set vpn ipsec logging log-modes 'any'
set vpn ipsec site-to-site peer 192.168.159.20 authentication id '192.168.159.10'
set vpn ipsec site-to-site peer 192.168.159.20 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 192.168.159.20 authentication pre-shared-secret '12345qwert'
set vpn ipsec site-to-site peer 192.168.159.20 connection-type 'initiate'
set vpn ipsec site-to-site peer 192.168.159.20 default-esp-group 'VTI'
set vpn ipsec site-to-site peer 192.168.159.20 ike-group 'VTI'
set vpn ipsec site-to-site peer 192.168.159.20 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 192.168.159.20 local-address '192.168.159.10'
set vpn ipsec site-to-site peer 192.168.159.20 vti bind 'vti0'
set vpn ipsec site-to-site peer 192.168.159.20 vti esp-group 'VTI'

vyos@vyos# run show vpn ipsec sa verbose 
Status of IKE charon daemon (strongSwan 5.7.2, Linux 4.19.28-amd64-vyos, x86_64):
  uptime: 96 minutes, since Mar 21 23:07:20 2019
  malloc: sbrk 1892352, mmap 0, used 827216, free 1065136
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 80
  loaded plugins: charon test-vectors ldap pkcs11 tpm aesni aes rc2 sha2 sha1 md5 mgf1 rdrand random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default connmark stroke vici updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp lookip error-notify certexpire led addrblock counters
Listening IP addresses:
  192.168.159.10
Connections:
peer-192.168.159.20-tunnel-vti:  192.168.159.10...192.168.159.20  IKEv1
peer-192.168.159.20-tunnel-vti:   local:  [192.168.159.10] uses pre-shared key authentication
peer-192.168.159.20-tunnel-vti:   remote: [192.168.159.20] uses pre-shared key authentication
peer-192.168.159.20-tunnel-vti:   child:  0.0.0.0/0 === 0.0.0.0/0 TUNNEL
Security Associations (1 up, 0 connecting):
peer-192.168.159.20-tunnel-vti[40]: ESTABLISHED 46 minutes ago, 192.168.159.10[192.168.159.10]...192.168.159.20[192.168.159.20]
peer-192.168.159.20-tunnel-vti[40]: IKEv1 SPIs: 19e276269acee286_i* 4fdbb7a2993805c1_r, pre-shared key reauthentication in 23 hours
peer-192.168.159.20-tunnel-vti[40]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
peer-192.168.159.20-tunnel-vti{2}:  REKEYED, TUNNEL, reqid 2, expires in 13 minutes
peer-192.168.159.20-tunnel-vti{2}:   0.0.0.0/0 === 0.0.0.0/0
peer-192.168.159.20-tunnel-vti{3}:  REKEYED, TUNNEL, reqid 2, expires in 13 minutes
peer-192.168.159.20-tunnel-vti{3}:   0.0.0.0/0 === 0.0.0.0/0
peer-192.168.159.20-tunnel-vti{4}:  INSTALLED, TUNNEL, reqid 2, ESP SPIs: c26f53d5_i dc5ce5be_o
peer-192.168.159.20-tunnel-vti{4}:  AES_CBC_256/HMAC_SHA1_96/MODP_2048, 2080 bytes_i (26 pkts, 8s ago), 1632 bytes_o (24 pkts, 8s ago), rekeying in 39 minutes
peer-192.168.159.20-tunnel-vti{4}:   0.0.0.0/0 === 0.0.0.0/0
2

An update:

setting the login level probably restarts vpn process.

“restart vpn” have the same effect of fixing the problem.

So I think there is something wrong with vpn.

3

I see you are using static routing over the VTI interface. But I think there is a bug in VyOS regarding this…

I don’t think the static routes are updated AFTER ipsec tunnel comes up. They are still listed in the route table, but this is incorrect. I have tried this with a rolling release about a month ago, and I could see with wireshark that vyos sent out traffic over the physical interface instead of the VTI when using ipsec…

This means, that you have to reinitate your routing statements after tunnels comes up. And since you’re using static routing, this won’t dynamically reiniate when your tunnel comes up.

If youre using dynamic routing like ospf on the vti, it should work.