IPsec VPN between VyOS 1.4 and Edgereouter

I-n-d-y · January 27, 2023, 7:34am

Hi,

I am trying to set up an IPsec VPN connection between two sites where each VPN router is behind NAT. The VPN connection was working when I used two Edgerouters but I want to move one site to VyOS router and get rid of the Edgerouter.

When I check the logs I can see that there is an issue during the authentication - the Edgerouter can not validate the identity of the VyoS router.

I followed the instructions in the documentation - copying the public RSA key to each peer. Maybe it is related to the different strongSwan versions used in each router’s firmware (Edgerouter: strongSwan swanctl 5.6.3 / VyOS: strongSwan swanctl 5.9.8).

SITE-A (VyOS router behind NAT)

configuration

set vpn ipsec esp-group IPSEC-ESP-DEFAULT lifetime '7200'
set vpn ipsec esp-group IPSEC-ESP-DEFAULT mode 'tunnel'
set vpn ipsec esp-group IPSEC-ESP-DEFAULT pfs 'enable'
set vpn ipsec esp-group IPSEC-ESP-DEFAULT proposal 1 encryption 'aes128'
set vpn ipsec esp-group IPSEC-ESP-DEFAULT proposal 1 hash 'sha1'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT dead-peer-detection action 'restart'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT dead-peer-detection interval '30'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT dead-peer-detection timeout '120'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT key-exchange 'ikev2'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT lifetime '14400'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT proposal 1 dh-group '14'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT proposal 1 encryption 'aes128'
set vpn ipsec ike-group IPSEC-IKE-DEFAULT proposal 1 hash 'sha1'
set vpn ipsec interface 'eth0.1204'
set vpn ipsec options disable-route-autoinstall
set vpn ipsec options interface 'eth0.1204'
set vpn ipsec site-to-site peer vpn-hn-wl-01 authentication local-id '@vpn-hn-hh-01-01'
set vpn ipsec site-to-site peer vpn-hn-wl-01 authentication mode 'rsa'
set vpn ipsec site-to-site peer vpn-hn-wl-01 authentication remote-id '@rt-hn-wl-02'
set vpn ipsec site-to-site peer vpn-hn-wl-01 authentication rsa local-key 'IPSEC-VPN-HN-HH-01-01'
set vpn ipsec site-to-site peer vpn-hn-wl-01 authentication rsa remote-key 'IPSEC-RT-HN-WL-02'
set vpn ipsec site-to-site peer vpn-hn-wl-01 connection-type 'initiate'
set vpn ipsec site-to-site peer vpn-hn-wl-01 default-esp-group 'IPSEC-ESP-DEFAULT'
set vpn ipsec site-to-site peer vpn-hn-wl-01 description 'VPN IPsec Peer RT-HN-WL-02'
set vpn ipsec site-to-site peer vpn-hn-wl-01 ike-group 'IPSEC-IKE-DEFAULT'
set vpn ipsec site-to-site peer vpn-hn-wl-01 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer vpn-hn-wl-01 local-address 'any'
set vpn ipsec site-to-site peer vpn-hn-wl-01 remote-address '<FQDN-SITE-B>'
set vpn ipsec site-to-site peer vpn-hn-wl-01 tunnel 1 local prefix '10.255.255.123/32'
set vpn ipsec site-to-site peer vpn-hn-wl-01 tunnel 1 remote prefix '10.255.255.151/32'

swanctl --list-conns

vpn-hn-wl-01: IKEv2, no reauthentication, rekeying every 14400s, dpd delay 30s
local: %any
remote: <FQDN-SITE-B>
local public key authentication:
id: vpn-hn-hh-01-01
certs: vpn-hn-hh-01-01
remote public key authentication:
id: rt-hn-wl-02
certs: rt-hn-wl-02
vpn-hn-wl-01-tunnel-1: TUNNEL, rekeying every 6545s, dpd action is start
local: 10.255.255.123/32
remote: 10.255.255.151/32

swanctl --initiate --ike vpn-hn-wl-01

[IKE] initiating IKE_SA vpn-hn-wl-01[9] to <PUBLIC-IP-SITE-B>
[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
[NET] sending packet: from 10.12.4.241[500] to <PUBLIC-IP-SITE-B>[500] (464 bytes)
[NET] received packet: from <PUBLIC-IP-SITE-B>[500] to 10.12.4.241[500] (464 bytes)
[ENC] parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(MULT_AUTH) ]
[CFG] selected proposal: IKE:AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_2048
[IKE] local host is behind NAT, sending keep alives
[IKE] remote host is behind NAT
[IKE] authentication of 'vpn-hn-hh-01-01' (myself) with RSA_EMSA_PKCS1_SHA2_256 successful
[ENC] generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(MOBIKE_SUP) N(NO_ADD_ADDR) N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
[NET] sending packet: from 10.12.4.241[4500] to <PUBLIC-IP-SITE-B>[4500] (444 bytes)
[NET] received packet: from <PUBLIC-IP-SITE-B>[4500] to 10.12.4.241[4500] (76 bytes)
[ENC] parsed IKE_AUTH response 1 [ N(AUTH_FAILED) ]
[IKE] received AUTHENTICATION_FAILED notify error
initiate failed: establishing IKE_SA 'vpn-hn-wl-01' failed

SITE-B (Edgerouter behind NAT)

configuration

set vpn ipsec auto-firewall-nat-exclude disable
set vpn ipsec esp-group IPSEC-ESP-DEFAULT compression disable
set vpn ipsec esp-group IPSEC-ESP-DEFAULT lifetime 7200
set vpn ipsec esp-group IPSEC-ESP-DEFAULT mode tunnel
set vpn ipsec esp-group IPSEC-ESP-DEFAULT pfs enable
set vpn ipsec esp-group IPSEC-ESP-DEFAULT proposal 1 encryption aes128
set vpn ipsec esp-group IPSEC-ESP-DEFAULT proposal 1 hash sha1
set vpn ipsec ike-group IPSEC-IKE-DEFAULT dead-peer-detection action restart
set vpn ipsec ike-group IPSEC-IKE-DEFAULT dead-peer-detection interval 30
set vpn ipsec ike-group IPSEC-IKE-DEFAULT dead-peer-detection timeout 120
set vpn ipsec ike-group IPSEC-IKE-DEFAULT ikev2-reauth no
set vpn ipsec ike-group IPSEC-IKE-DEFAULT key-exchange ikev2
set vpn ipsec ike-group IPSEC-IKE-DEFAULT lifetime 14400
set vpn ipsec ike-group IPSEC-IKE-DEFAULT proposal 1 dh-group 14
set vpn ipsec ike-group IPSEC-IKE-DEFAULT proposal 1 encryption aes128
set vpn ipsec ike-group IPSEC-IKE-DEFAULT proposal 1 hash sha1
set vpn ipsec ipsec-interfaces interface eth0
set vpn ipsec nat-traversal enable
set vpn ipsec site-to-site peer <FQDN-SITE-A> authentication id @rt-hn-wl-02
set vpn ipsec site-to-site peer <FQDN-SITE-A> authentication mode rsa
set vpn ipsec site-to-site peer <FQDN-SITE-A> authentication remote-id @vpn-hn-hh-01-01
set vpn ipsec site-to-site peer <FQDN-SITE-A> authentication rsa-key-name IPSEC-VPN-HN-HH-01-01
set vpn ipsec site-to-site peer <FQDN-SITE-A> connection-type respond
set vpn ipsec site-to-site peer <FQDN-SITE-A> default-esp-group IPSEC-ESP-DEFAULT
set vpn ipsec site-to-site peer <FQDN-SITE-A> description 'VPN IPsec Peer VPN-HN-HH-01-01'
set vpn ipsec site-to-site peer <FQDN-SITE-A> ike-group IPSEC-IKE-DEFAULT
set vpn ipsec site-to-site peer <FQDN-SITE-A> ikev2-reauth inherit
set vpn ipsec site-to-site peer <FQDN-SITE-A> local-address any
set vpn ipsec site-to-site peer <FQDN-SITE-A> tunnel 1 allow-nat-networks disable
set vpn ipsec site-to-site peer <FQDN-SITE-A> tunnel 1 allow-public-networks disable
set vpn ipsec site-to-site peer <FQDN-SITE-A> tunnel 1 local prefix 10.255.255.151/32
set vpn ipsec site-to-site peer <FQDN-SITE-A> tunnel 1 remote prefix 10.255.255.123/32

swanctl --list-conns

peer-<FQDN-SITE-A>-tunnel-1: , no reauthentication, no rekeying
local: %any
remote: <FQDN-SITE-A>
local public key authentication:
id: rt-hn-wl-02
certs: rt-hn-wl-02
remote public key authentication:
id: vpn-hn-hh-01-01
certs: vpn-hn-hh-01-01
peer-<FQDN-SITE-A>-tunnel-1: TUNNEL, rekeying every 6660s
local: 10.255.255.151/32
remote: 10.255.255.123/32

swanctl --log

09[NET] received packet: from <PUBLIC-IP-SITE-A>[62852] to 172.16.150.253[500] (464 bytes)
09[ENC] parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
09[IKE] <PUBLIC-IP-SITE-A> is initiating an IKE_SA
09[IKE] local host is behind NAT, sending keep alives
09[IKE] remote host is behind NAT
09[ENC] generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(MULT_AUTH) ]
09[NET] sending packet: from 172.16.150.253[500] to <PUBLIC-IP-SITE-A>[62852] (464 bytes)
08[NET] received packet: from <PUBLIC-IP-SITE-A>[62853] to 172.16.150.253[4500] (444 bytes)
08[ENC] parsed IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(MOBIKE_SUP) N(NO_ADD_ADDR) N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
08[CFG] looking for peer configs matching 172.16.150.253[rt-hn-wl-02]...<PUBLIC-IP-SITE-A>[vpn-hn-hh-01-01]
08[CFG] selected peer config 'peer-<FQDN-SITE-A>-tunnel-1'
08[CFG] using trusted certificate "vpn-hn-hh-01-01"
08[IKE] signature validation failed, looking for another key
08[IKE] peer supports MOBIKE
08[ENC] generating IKE_AUTH response 1 [ N(AUTH_FAILED) ]
08[NET] sending packet: from 172.16.150.253[4500] to <PUBLIC-IP-SITE-A>[62853] (76 bytes)

Any help or idea is appreciated.

Thanks,

Lars

a.srividya · January 27, 2023, 6:46pm

Could you please test the ipsec connection in the latest rolling version for both the devices

I-n-d-y · January 29, 2023, 1:53pm

I have installed the latest VyOS rolling version on my router (1.4-rolling-202301290317). I can not update the Edgerouter as it is already on the latest firmware version. Unfortunately, Ubiquiti is not very up to date with the software versions in their firmware.

The behaviour is the same with the latest VyOS version. The Edgerouter can not validate the signature of the VyOS router’s public key. I have already check the public keys and they are correct on both routers.

Can I somehow enable debbugging in strongSwan to better see what is goning on during the valudation phase?

Thanks.

a.srividya · January 30, 2023, 6:15pm

Can you please attach the charon logs here by running the following command:

sudo journalctl -b /usr/lib/ipsec/charon > /tmp/ipsec.log

I-n-d-y · February 1, 2023, 7:24am

Here is the log file from my router running VyOS 1.4-rolling-202301290317 (SITE-A): SITE-A_ipsec.log (7.2 KB)

Unfortunately, the peer router (Edgerouter) on SITE-B who seems to have the issue with the signature validation does not have any journal log files. /var/log/charon.log only contains a few entries like this:

Jan 24 20:17:14 00[DMN] signal of type SIGINT received. Shutting down
Jan 24 20:17:16 00[DMN] Starting IKE charon daemon (strongSwan 5.6.3, Linux 4.9.79-UBNT, mips64)
Jan 24 20:18:52 11[IKE] <1> <PUBLIC-IP-SITE-A> is initiating an IKE_SA
Jan 27 08:00:57 09[IKE] <11> <PUBLIC-IP-SITE-A> is initiating an IKE_SA
Jan 29 14:29:32 10[IKE] <27> <PUBLIC-IP-SITE-A> is initiating an IKE_SA
Jan 29 14:32:23 16[IKE] <28> <PUBLIC-IP-SITE-A> is initiating an IKE_SA

Thanks.

Viacheslav · February 1, 2023, 7:47am

Is it working with PSK?

I-n-d-y · February 1, 2023, 6:29pm

Both VPN routers (VyOS and Edgerouter) are behind ISP routers which have dynamic IP addresses and use NAT, so regarding the docs I can not use PSK, right?

I-n-d-y · February 4, 2023, 12:51pm

I did some more troubleshooting and I think I have found the cause for the validation issue. When I run swanctl --list-certs on both routers, the output is different.

SITE-A (VyOS router):

SITE-A:~# swanctl --list-certs

List of Raw Public Keys

  subject:  "SITE-A"
  pubkey:    RSA 2048 bits, has private key
  keyid:     5c:1d:2c:07:fc:20:71:c6:b0:87:93:af:37:93:4a:51:38:b1:ff:6e
  subjkey:   62:4a:90:1d:dd:45:fc:9a:d0:45:fd:34:97:92:b2:b0:cf:2d:42:76

  subject:  "SITE-B"
  pubkey:    RSA 2048 bits
  keyid:     c5:0d:c0:b1:9b:92:2b:09:a0:c3:9d:db:e2:47:41:fa:cd:58:f0:1c
  subjkey:   87:40:5e:2d:69:1c:4f:df:04:91:e2:c3:b9:85:02:36:a0:5e:86:d2

SITE-B (Edgerouter):

SITE-B:~# swanctl --list-certs

List of Raw Public Keys

  subject:  "SITE-A"
  pubkey:    RSA 2072 bits
  keyid:     1b:73:17:66:4f:c4:6b:89:53:ad:e9:ec:8c:78:34:9c:75:f7:cf:b3
  subjkey:   93:47:41:c5:d7:ba:fa:36:5d:60:f0:94:d7:ef:15:6c:d9:1b:a2:b9

  subject:  "SITE-B"
  pubkey:    RSA 2048 bits, has private key
  keyid:     c5:0d:c0:b1:9b:92:2b:09:a0:c3:9d:db:e2:47:41:fa:cd:58:f0:1c
  subjkey:   87:40:5e:2d:69:1c:4f:df:04:91:e2:c3:b9:85:02:36:a0:5e:86:d2

The Edgerouter on SITE-B somehow modifies SITE-A’s public key during commit. When I compare the keys on both routers using the show command, they are exactly the same.

I will write a post in the Ubiquiti forum as it seems related to the Edgerouter OS.

I-n-d-y · February 8, 2023, 7:07am

Since I received no response from the Ubiquiti community regarding the IPsec RSA key issue, I decided to change my IPsec configuration to use x509 certificates for authentication.

I took me a while as well to get everything together that is needed but now I can successfully establish an IPsec connection between the two routers.

dannytech · March 19, 2023, 6:23pm

You’re not alone, I just ran into this issue and this post matches exactly the issue I’m having. I managed to get a little bit further, but it’s still not working for me. I noticed that the generate vpn rsa-key bits 2048 command on the EdgeRouter output a public key starting with 0sAw rather than MIIB, so it appears they’re storing it in a slightly different format. I generated the key on the EdgeRouter and then transferred the private key to VyOS, and added the 0sAw public key to the EdgeRouter. Now, swanctl --list-certs shows a 2048-bit key, but the keyid doesn’t match for whatever reason. Also, the file /etc/ipsec.d/certs/<remote-key>.pub contains the remote public key, exactly matching what is on VyOS, so it seems that while Strongswan is loading the key, it does something that causes it to not match?

pepe · March 19, 2023, 7:39pm

VyOS PKI store all data in PKCS#8 format (https://docs.vyos.io/en/latest/configuration/pki/index.html#pki).
So you must convert key generated by ER.

dannytech · March 19, 2023, 8:01pm

Yeah, it works now. Turns out VyOS hadn’t picked up the changes, a quick reboot fixed this and the tunnel now establishes.

I-n-d-y · March 20, 2023, 6:58am

Glad to hear that it is working now. My workaround was to use X509 certificates instead of RSA keys.
What commands did you use to convert the keys? Thanks.

dannytech · March 20, 2023, 1:15pm

I wasn’t sure what format EdgeOS was expecting so I just generated the key on the EdgeRouter instead, since it outputs the correct format. I then transferred the private key to the VyOS router and used openssl rsa -in private.key -pubout to get the public key in the format VyOS expects.

I-n-d-y · March 23, 2023, 9:42am

Thanks. I’ll try moving back to RSA keys too, so that I don’t have to deal with the certificates.