VyOS IPSec Site to Site with OpenSwan

Hi there,

I have been trying to solve this annoying issue for a few days now, basically we have a few servers in different data centres, which handle all of our internal network routing. They’re all interconnected by IPSec Site-to-Site VPNs so that one data centre private network can talk to another.

We have found this to be fairly unreliable, so we’re moving to VyOS. The problem is, we need to do this in stages, so replace one server with VyOS, have it connect to the other servers, replace the next one, etc.

Our current OpenSwan servers run CentOS 5.11 with OpenSwan 2.6.32.

For some strange reason, I cannot get our VyOS installation to connect to the OpenSwan servers over IPSec, we continually get the following messages:

==VYOS==

Jan 21 19:02:22 vyos pluto[10356]: packet from :500: ignoring Vendor ID payload [4f4568794c64414365636661]
Jan 21 19:02:22 vyos pluto[10356]: packet from :500: received Vendor ID payload [Dead Peer Detection]
Jan 21 19:02:22 vyos pluto[10356]: packet from :500: ignoring Vendor ID payload [RFC 3947]
Jan 21 19:02:22 vyos pluto[10356]: packet from :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-03]
Jan 21 19:02:22 vyos pluto[10356]: packet from :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n]
Jan 21 19:02:22 vyos pluto[10356]: packet from :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02]
Jan 21 19:02:22 vyos pluto[10356]: packet from :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-00]
Jan 21 19:02:22 vyos pluto[10356]: “peer--tunnel-1” #4630: responding to Main Mode
Jan 21 19:02:22 vyos pluto[10356]: “peer--tunnel-1” #4630: Oakley Transform [AES_CBC (128), HMAC_SHA1, MODP_1536] refused due to strict flag
Jan 21 19:02:22 vyos pluto[10356]: “peer--tunnel-1” #4630: Oakley Transform [AES_CBC (128), HMAC_SHA1, MODP_1024] refused due to strict flag
Jan 21 19:02:22 vyos pluto[10356]: “peer--tunnel-1” #4630: no acceptable Oakley Transform
Jan 21 19:02:22 vyos pluto[10356]: “peer--tunnel-1” #4630: sending notification NO_PROPOSAL_CHOSEN to :500
Jan 21 19:02:37 vyos pluto[10356]: “peer--tunnel-1” #4628: max number of retransmissions (2) reached STATE_QUICK_I1. No acceptable response to our first Quick Mode message: perhaps peer likes no proposal
Jan 21 19:02:37 vyos pluto[10356]: “peer--tunnel-1” #4628: starting keying attempt 5 of an unlimited number
Jan 21 19:02:37 vyos pluto[10356]: “peer--tunnel-1” #4631: initiating Quick Mode PSK+ENCRYPT+TUNNEL+UP to replace #4628 {using isakmp#4619}
Jan 21 19:02:37 vyos pluto[10356]: “peer--tunnel-1” #4619: ignoring informational payload, type INVALID_ID_INFORMATION

==OPENSWAN==

Jan 21 10:50:50 openswan pluto[5179]: “vyos/1x1” #59: sending encrypted notification INVALID_ID_INFORMATION to :500
Jan 21 10:50:55 openswan pluto[5179]: “vyos/1x1” #4: ignoring informational payload, type NO_PROPOSAL_CHOSEN msgid=00000000
Jan 21 10:50:55 openswan pluto[5179]: “vyos/1x1” #4: received and ignored informational message
Jan 21 10:51:00 openswan pluto[5179]: “vyos/1x1” #59: the peer proposed: 10.10.8.0/24:0/0 -> 172.16.1.0/24:0/0
Jan 21 10:51:00 openswan pluto[5179]: “vyos/1x1” #59: cannot respond to IPsec SA request because no connection is known for 10.10.8.0/24===<>[+S=C]…<>[+S=C]===172.16.1.0/24

So phase 1 comes up fine, it’s only phase 2 / ESP that won’t come up.

All of my searching online has lead me to believe there is a configuration mismatch between the two based on the errors we’re receiving, but I have tried heaps of different combinations, none of which have worked, plus the fact that the settings look almost identical.

Our configs for both are:

==VYOS==

set vpn ipsec esp-group ESP-1W lifetime ‘1800’
set vpn ipsec esp-group ESP-1W pfs ‘disable’
set vpn ipsec esp-group ESP-1W proposal 1 encryption ‘aes128’
set vpn ipsec esp-group ESP-1W proposal 1 hash ‘sha1’
set vpn ipsec ike-group IKE-1W lifetime ‘3600’
set vpn ipsec ike-group IKE-1W proposal 1 dh-group ‘14’
set vpn ipsec ike-group IKE-1W proposal 1 encryption ‘aes128’
set vpn ipsec ike-group IKE-1W proposal 1 hash ‘sha1’
set vpn ipsec ipsec-interfaces interface ‘eth0’
set vpn ipsec site-to-site peer authentication mode ‘pre-shared-secret’
set vpn ipsec site-to-site peer authentication pre-shared-secret ‘’
set vpn ipsec site-to-site peer connection-type ‘initiate’
set vpn ipsec site-to-site peer default-esp-group ‘ESP-1W’
set vpn ipsec site-to-site peer ike-group ‘IKE-1W’
set vpn ipsec site-to-site peer local-address ‘’
set vpn ipsec site-to-site peer tunnel 1 allow-nat-networks ‘disable’
set vpn ipsec site-to-site peer tunnel 1 esp-group ‘ESP-1W’
set vpn ipsec site-to-site peer tunnel 1 local prefix ‘172.16.1.0/24’
set vpn ipsec site-to-site peer tunnel 1 remote prefix ‘10.10.8.0/24’

==OPENSWAN==

conn vyos
#ADDRESSING
left=
right=
leftsourceip=
leftsubnets={172.16.10.0/24}
rightsubnets={10.10.8.0/24}

    #PHASE 1
    ikelifetime=1h
    ike=aes128-sha1

    #PHASE 2
    phase2=esp
    phase2alg=aes128-sha1;modp2048
    authby=secret
    type=tunnel
    auto=start
    pfs=no
    forceencaps=yes

Obviously public IPs have been replaced with and above.

Any help / suggestions anyone can give would be greatly appreciated. :slight_smile:

Thanks in advance.

I would test with a different DH group like try dh-group ‘2’ on vyos and modp1024 on openswan
It appears in the error that one of them is offering DH group 5 and 2 but not 14

Thanks for the suggestion, I’ve changed the dh-group to ‘2’ and set modp1024, however it still doesn’t seem to be working:

==VYOS==

set vpn ipsec esp-group ESP-1W lifetime ‘1800’
set vpn ipsec esp-group ESP-1W pfs ‘disable’
set vpn ipsec esp-group ESP-1W proposal 1 encryption ‘aes128’
set vpn ipsec esp-group ESP-1W proposal 1 hash ‘sha1’
set vpn ipsec ike-group IKE-1W lifetime ‘3600’
set vpn ipsec ike-group IKE-1W proposal 1 dh-group ‘2’
set vpn ipsec ike-group IKE-1W proposal 1 encryption ‘aes128’
set vpn ipsec ike-group IKE-1W proposal 1 hash ‘sha1’

==OPENSWAN==

    #PHASE 2
    phase2=esp
    phase2alg=aes128-sha1;modp1024
    authby=secret
    type=tunnel
    auto=start
    pfs=no
    forceencaps=yes

Logs:

==VYOS==

Jan 21 22:04:07 vyos pluto[10356]: “peer--tunnel-1” #5061: sending encrypted notification INVALID_ID_INFORMATION to :500
Jan 21 22:04:17 vyos pluto[10356]: “peer--tunnel-1” #5061: Quick Mode I1 message is unacceptable because it uses a previously used Message ID 0xd0b1ae67 (perhaps this is a duplicated packet)
Jan 21 22:04:17 vyos pluto[10356]: “peer--tunnel-1” #5061: sending encrypted notification INVALID_MESSAGE_ID to :500
Jan 21 22:04:21 vyos pluto[10356]: “peer--tunnel-1” #5061: ignoring informational payload, type INVALID_ID_INFORMATION
Jan 21 22:04:37 vyos pluto[10356]: “peer--tunnel-1” #5061: Quick Mode I1 message is unacceptable because it uses a previously used Message ID 0xd0b1ae67 (perhaps this is a duplicated packet)
Jan 21 22:04:37 vyos pluto[10356]: “peer--tunnel-1” #5061: sending encrypted notification INVALID_MESSAGE_ID to :500
Jan 21 22:05:01 vyos pluto[10356]: “peer--tunnel-1” #5068: max number of retransmissions (2) reached STATE_QUICK_I1. No acceptable response to our fopenswant Quick Mode message: perhaps peer likes no proposal
Jan 21 22:05:01 vyos pluto[10356]: “peer--tunnel-1” #5068: starting keying attempt 9 of an unlimited number
Jan 21 22:05:01 vyos pluto[10356]: “peer--tunnel-1” #5069: initiating Quick Mode PSK+ENCRYPT+TUNNEL+UP to replace #5068 {using isakmp#5061}
Jan 21 22:05:01 vyos pluto[10356]: “peer--tunnel-1” #5061: ignoring informational payload, type INVALID_ID_INFORMATION
Jan 21 22:05:11 vyos pluto[10356]: “peer--tunnel-1” #5061: ignoring informational payload, type INVALID_ID_INFORMATION
Jan 21 22:05:17 vyos pluto[10356]: “peer--tunnel-1” #5061: cannot respond to IPsec SA request because no connection is known for 172.16.10.0/24===[]…[]===10.10.8.0/24

==OPENSWAN==

Jan 21 13:52:20 openswan pluto[6743]: “vyos/1x1” #77: starting keying attempt 10 of an unlimited number
Jan 21 13:52:20 openswan pluto[6743]: “vyos/1x1” #79: initiating Quick Mode PSK+ENCRYPT+TUNNEL+UP+IKEv2ALLOW+SAREFTRACK to replace #77 {using isakmp#4 msgid:d79ef699 proposal=AES(12)_128-SHA1(2)_160 pfsgroup=no-pfs}
Jan 21 13:52:20 openswan pluto[6743]: “vyos/1x1” #4: ignoring informational payload, type INVALID_ID_INFORMATION msgid=00000000
Jan 21 13:52:20 openswan pluto[6743]: “vyos/1x1” #4: received and ignored informational message
Jan 21 13:52:30 openswan pluto[6743]: “vyos/1x1” #4: ignoring informational payload, type INVALID_MESSAGE_ID msgid=00000000
Jan 21 13:52:30 openswan pluto[6743]: “vyos/1x1” #4: received and ignored informational message
Jan 21 13:52:34 openswan pluto[6743]: “vyos/1x1” #4: the peer proposed: 10.10.8.0/24:0/0 -> 172.16.1.0/24:0/0
Jan 21 13:52:34 openswan pluto[6743]: “vyos/1x1” #4: cannot respond to IPsec SA request because no connection is known for 10.10.8.0/24===<>[+S=C]…<>[+S=C]===172.16.1.0/24
Jan 21 13:52:34 openswan pluto[6743]: “vyos/1x1” #4: sending encrypted notification INVALID_ID_INFORMATION to :500
Jan 21 13:52:50 openswan pluto[6743]: “vyos/1x1” #4: ignoring informational payload, type INVALID_MESSAGE_ID msgid=00000000
Jan 21 13:52:50 openswan pluto[6743]: “vyos/1x1” #4: received and ignored informational message

Hi,

First, try to change in default lifetime in both phase:
My config:

show vpn ipsec esp-group ESP compression disable lifetime 3600 mode tunnel pfs disable proposal 1 { encryption aes256 hash sha1 }

show vpn ipsec ike-group IKE dead-peer-detection { action clear interval 30 timeout 120 } lifetime 86400 proposal 1 { dh-group 5 encryption aes256 hash sha1 }

Second, you may try add remote-id param to site:

Third,

I know is stupid, but sometimes i must restart ipsec process (or system) in vyos to able connect to difrent site (specially if hardware is difrent from VyOS)

P.s Sorry for my english.

Thanks for your help, I’ve tried those three solutions, but I’m still not seeing the tunnel come up:

==VYOS==

set vpn ipsec esp-group ESP-1W compression ‘disable’
set vpn ipsec esp-group ESP-1W lifetime ‘3600’
set vpn ipsec esp-group ESP-1W mode ‘tunnel’
set vpn ipsec esp-group ESP-1W pfs ‘disable’
set vpn ipsec esp-group ESP-1W proposal 1 encryption ‘aes128’
set vpn ipsec esp-group ESP-1W proposal 1 hash ‘sha1’
set vpn ipsec ike-group IKE-1W ikev2-reauth ‘no’
set vpn ipsec ike-group IKE-1W key-exchange ‘ikev1’
set vpn ipsec ike-group IKE-1W lifetime ‘86400’
set vpn ipsec ike-group IKE-1W proposal 1 dh-group ‘2’
set vpn ipsec ike-group IKE-1W proposal 1 encryption ‘aes128’
set vpn ipsec ike-group IKE-1W proposal 1 hash ‘sha1’
set vpn ipsec ipsec-interfaces interface ‘eth0’
set vpn ipsec site-to-site peer authentication mode ‘pre-shared-secret’
set vpn ipsec site-to-site peer authentication pre-shared-secret ‘’
set vpn ipsec site-to-site peer authentication remote-id ‘’
set vpn ipsec site-to-site peer connection-type ‘initiate’
set vpn ipsec site-to-site peer default-esp-group ‘ESP-1W’
set vpn ipsec site-to-site peer ike-group ‘IKE-1W’
set vpn ipsec site-to-site peer ikev2-reauth ‘inherit’
set vpn ipsec site-to-site peer local-address ‘’
set vpn ipsec site-to-site peer tunnel 1 allow-nat-networks ‘disable’
set vpn ipsec site-to-site peer tunnel 1 allow-public-networks ‘disable’
set vpn ipsec site-to-site peer tunnel 1 esp-group ‘ESP-1W’
set vpn ipsec site-to-site peer tunnel 1 local prefix ‘172.16.1.0/24’
set vpn ipsec site-to-site peer tunnel 1 remote prefix ‘10.10.8.0/24’

I also tried rebooting the entire server, as well as just the VPN service.

Does anyone else have any suggestions?

Does anyone else have any suggestions?