Site to Site dmvpn and Spoek Site behind NAT

General questions
1

Hi

I want to setup a site to site dmvpn on 1.2.0 rc10.
In the spock site used pppoe over nat connect to internet.
in the hub site used static public IP connect to Internet.

the Vyos setup follow information

Hub Site setup：(ver 1.2.0 rc10)

ethernet eth0 {
address 116.90.86.181/24
duplex auto
hw-id 00:50:56:95:6e:1a
smp-affinity auto
speed auto
}
ethernet eth1 {
address 172.16.101.1/24
duplex auto
hw-id 00:50:56:95:8e:c3
smp-affinity auto
speed auto
}
loopback lo {
}
tunnel tun0 {
address 10.0.0.1/24
encapsulation gre
local-ip 116.90.86.181
multicast enable
parameters {
ip {
key 1
}
}
}

nhrp {
tunnel tun0 {
cisco-authentication
holding-time 300
multicast dynamic
redirect
}
}
static {
route 0.0.0.0/0 {
next-hop 116.90.86.254 {
}
}
route 192.168.101.0/24 {
next-hop 10.0.0.2 {
}
}
}

ipsec {
esp-group ESP-HUB {
compression disable
lifetime 1800
mode tunnel
pfs dh-group2
proposal 1 {
encryption aes256
hash sha256
}
proposal 2 {
encryption 3des
hash md5
}
}
ike-group IKE-HUB {
ikev2-reauth no
key-exchange ikev1
lifetime 3600
proposal 1 {
dh-group 2
encryption aes256
hash sha1
}
proposal 2 {
dh-group 2
encryption aes128
hash sha1
}
}
ipsec-interfaces {
interface eth0
}
nat-traversal enable
profile IDC-VPN {
authentication {
mode pre-shared-secret
pre-shared-secret
}
bind {
tunnel tun0
}
esp-group ESP-HUB
ike-group IKE-HUB
}
}
And Spock Site setup (ver 1.2.0 rc10)

ethernet eth0 {
duplex auto
hw-id 00:e0:67:08:81:44
pppoe 0 {
default-route auto
mtu 1492
name-server auto
password xxx
user-id xxx
}
smp-affinity auto
speed auto
}
ethernet eth1 {
duplex auto
hw-id 00:e0:67:08:81:45
smp-affinity auto
speed auto
}
ethernet eth2 {
duplex auto
hw-id 00:e0:67:08:81:46
smp-affinity auto
speed auto
}
ethernet eth3 {
address 192.168.101.1/24
duplex auto
hw-id 00:e0:67:08:81:47
smp-affinity auto
speed auto
}
loopback lo {
}
tunnel tun0 {
address 10.0.0.2/24
encapsulation gre
local-ip 0.0.0.0
multicast enable
parameters {
ip {
key 1
}
}
}
nhrp {
tunnel tun0 {
cisco-authentication
map 10.0.0.1/24 {
nbma-address 116.90.86.181
register
}
multicast nhs
redirect
shortcut
}
}
static {
route 172.16.101.0/24 {
next-hop 10.0.0.1 {
}
}
}
ipsec {
esp-group ESP-SPOKE {
compression disable
lifetime 1800
mode tunnel
pfs dh-group2
proposal 1 {
encryption aes256
hash sha256
}
proposal 2 {
encryption 3des
hash md5
}
}
ike-group IKE-SPOKE {
ikev2-reauth no
key-exchange ikev1
lifetime 3600
proposal 1 {
dh-group 2
encryption aes256
hash sha1
}
proposal 2 {
dh-group 2
encryption aes128
hash sha1
}
}
ipsec-interfaces {
interface pppoe0
}
nat-traversal enable
profile IDC-ZZ {
authentication {
mode pre-shared-secret
pre-shared-secret
}
bind {
tunnel tun0
}
esp-group ESP-SPOKE
ike-group IKE-SPOKE
}
}

I check log see follow info

In Hub show log all | grep charon

Dec 9 13:02:00 vyos charon: 08[ENC] generating INFORMATIONAL_V1 request 3953897240 [ HASH N(INVAL_ID) ]
Dec 9 13:02:00 vyos charon: 08[NET] sending packet: from 116.90.86.181[4500] to 115.60.57.13[23132] (76 bytes)
Dec 9 13:04:57 vyos charon: 10[NET] received packet: from 115.60.57.13[23132] to 116.90.86.181[4500] (92 bytes)
Dec 9 13:04:57 vyos charon: 10[ENC] parsed INFORMATIONAL_V1 request 1310358166 [ HASH D ]
Dec 9 13:04:57 vyos charon: 10[IKE] received DELETE for IKE_SA vpnprof-dmvpn-tun0[116]
Dec 9 13:04:57 vyos charon: 10[IKE] deleting IKE_SA vpnprof-dmvpn-tun0[116] between 116.90.86.181[116.90.86.181]…115.60.57.13[100.64.21.35]
Dec 9 13:04:57 vyos charon: 14[NET] received packet: from 115.60.57.13[21532] to 116.90.86.181[500] (216 bytes)
Dec 9 13:04:57 vyos charon: 14[ENC] parsed ID_PROT request 0 [ SA V V V V V ]
Dec 9 13:04:57 vyos charon: 14[IKE] received XAuth vendor ID
Dec 9 13:04:57 vyos charon: 14[IKE] received DPD vendor ID
Dec 9 13:04:57 vyos charon: 14[IKE] received FRAGMENTATION vendor ID
Dec 9 13:04:57 vyos charon: 14[IKE] received NAT-T (RFC 3947) vendor ID
Dec 9 13:04:57 vyos charon: 14[IKE] received draft-ietf-ipsec-nat-t-ike-02\n vendor ID
Dec 9 13:04:57 vyos charon: 14[IKE] 115.60.57.13 is initiating a Main Mode IKE_SA
Dec 9 13:04:57 vyos charon: 14[ENC] generating ID_PROT response 0 [ SA V V V V ]
Dec 9 13:04:57 vyos charon: 14[NET] sending packet: from 116.90.86.181[500] to 115.60.57.13[21532] (160 bytes)
Dec 9 13:04:57 vyos charon: 15[NET] received packet: from 115.60.57.13[21532] to 116.90.86.181[500] (244 bytes)
Dec 9 13:04:57 vyos charon: 15[ENC] parsed ID_PROT request 0 [ KE No NAT-D NAT-D ]
Dec 9 13:04:57 vyos charon: 15[IKE] remote host is behind NAT
Dec 9 13:04:57 vyos charon: 15[ENC] generating ID_PROT response 0 [ KE No NAT-D NAT-D ]
Dec 9 13:04:57 vyos charon: 15[NET] sending packet: from 116.90.86.181[500] to 115.60.57.13[21532] (244 bytes)
Dec 9 13:04:57 vyos charon: 13[NET] received packet: from 115.60.57.13[23132] to 116.90.86.181[4500] (76 bytes)
Dec 9 13:04:57 vyos charon: 13[ENC] parsed ID_PROT request 0 [ ID HASH ]
Dec 9 13:04:57 vyos charon: 13[CFG] looking for pre-shared key peer configs matching 116.90.86.181…115.60.57.13[100.64.21.35]
Dec 9 13:04:57 vyos charon: 13[CFG] selected peer config “vpnprof-dmvpn-tun0”
Dec 9 13:04:57 vyos charon: 13[IKE] IKE_SA vpnprof-dmvpn-tun0[117] established between 116.90.86.181[116.90.86.181]…115.60.57.13[100.64.21.35]
Dec 9 13:04:57 vyos charon: 13[IKE] scheduling rekeying in 3588s
Dec 9 13:04:57 vyos charon: 13[IKE] maximum IKE_SA lifetime 3948s
Dec 9 13:04:57 vyos charon: 13[ENC] generating ID_PROT response 0 [ ID HASH ]
Dec 9 13:04:57 vyos charon: 13[NET] sending packet: from 116.90.86.181[4500] to 115.60.57.13[23132] (76 bytes)
Dec 9 13:04:57 vyos charon: 07[NET] received packet: from 115.60.57.13[23132] to 116.90.86.181[4500] (332 bytes)
Dec 9 13:04:57 vyos charon: 07[ENC] parsed QUICK_MODE request 614827736 [ HASH SA No KE ID ID ]
Dec 9 13:04:57 vyos charon: 07[IKE] no matching CHILD_SA config found

In Spock site show log all | grep charon

Dec 9 13:05:13 vyos charon: 07[CFG] vici terminate with source me 100.64.21.35 and other 116.90.86.181
Dec 9 13:05:13 vyos charon: 06[IKE] deleting IKE_SA vpnprof-dmvpn-tun0[38] between 100.64.21.35[100.64.21.35]…116.90.86.181[116.90.86.181]
Dec 9 13:05:13 vyos charon: 06[IKE] sending DELETE for IKE_SA vpnprof-dmvpn-tun0[38]
Dec 9 13:05:13 vyos charon: 06[ENC] generating INFORMATIONAL_V1 request 1310358166 [ HASH D ]
Dec 9 13:05:13 vyos charon: 06[NET] sending packet: from 100.64.21.35[4500] to 116.90.86.181[4500] (92 bytes)
Dec 9 13:05:13 vyos charon: 06[CFG] vici initiate ‘dmvpn’, me 100.64.21.35, other 116.90.86.181, limits 0
Dec 9 13:05:13 vyos charon: 07[IKE] initiating Main Mode IKE_SA vpnprof-dmvpn-tun0[39] to 116.90.86.181
Dec 9 13:05:13 vyos charon: 07[ENC] generating ID_PROT request 0 [ SA V V V V V ]
Dec 9 13:05:13 vyos charon: 07[NET] sending packet: from 100.64.21.35[500] to 116.90.86.181[500] (216 bytes)
Dec 9 13:05:13 vyos charon: 05[NET] received packet: from 116.90.86.181[500] to 100.64.21.35[500] (160 bytes)
Dec 9 13:05:13 vyos charon: 05[ENC] parsed ID_PROT response 0 [ SA V V V V ]
Dec 9 13:05:13 vyos charon: 05[IKE] received XAuth vendor ID
Dec 9 13:05:13 vyos charon: 05[IKE] received DPD vendor ID
Dec 9 13:05:13 vyos charon: 05[IKE] received FRAGMENTATION vendor ID
Dec 9 13:05:13 vyos charon: 05[IKE] received NAT-T (RFC 3947) vendor ID
Dec 9 13:05:13 vyos charon: 05[ENC] generating ID_PROT request 0 [ KE No NAT-D NAT-D ]
Dec 9 13:05:13 vyos charon: 05[NET] sending packet: from 100.64.21.35[500] to 116.90.86.181[500] (244 bytes)
Dec 9 13:05:13 vyos charon: 07[NET] received packet: from 116.90.86.181[500] to 100.64.21.35[500] (244 bytes)
Dec 9 13:05:13 vyos charon: 07[ENC] parsed ID_PROT response 0 [ KE No NAT-D NAT-D ]
Dec 9 13:05:13 vyos charon: 07[IKE] local host is behind NAT, sending keep alives
Dec 9 13:05:13 vyos charon: 07[ENC] generating ID_PROT request 0 [ ID HASH ]
Dec 9 13:05:13 vyos charon: 07[NET] sending packet: from 100.64.21.35[4500] to 116.90.86.181[4500] (76 bytes)
Dec 9 13:05:13 vyos charon: 13[NET] received packet: from 116.90.86.181[4500] to 100.64.21.35[4500] (76 bytes)
Dec 9 13:05:13 vyos charon: 13[ENC] parsed ID_PROT response 0 [ ID HASH ]
Dec 9 13:05:13 vyos charon: 13[IKE] IKE_SA vpnprof-dmvpn-tun0[39] established between 100.64.21.35[100.64.21.35]…116.90.86.181[116.90.86.181]
Dec 9 13:05:13 vyos charon: 13[IKE] scheduling rekeying in 3304s
Dec 9 13:05:13 vyos charon: 13[IKE] maximum IKE_SA lifetime 3664s
Dec 9 13:05:13 vyos charon: 13[ENC] generating QUICK_MODE request 614827736 [ HASH SA No KE ID ID ]
Dec 9 13:05:13 vyos charon: 13[NET] sending packet: from 100.64.21.35[4500] to 116.90.86.181[4500] (332 bytes)
Dec 9 13:05:13 vyos charon: 04[NET] received packet: from 116.90.86.181[4500] to 100.64.21.35[4500] (76 bytes)
Dec 9 13:05:13 vyos charon: 04[ENC] parsed INFORMATIONAL_V1 request 3550378600 [ HASH N(INVAL_ID) ]
Dec 9 13:05:13 vyos charon: 04[IKE] received INVALID_ID_INFORMATION error notify

Please help me fix it

Thanks
David

3

Today I change vpn log to level 2, see follow info.

In Spock Site:

Dec 10 05:05:59 vyos charon[12687]: 13[CFG] proposing traffic selectors for us:
Dec 10 05:05:59 vyos charon[12687]: 13[CFG] 100.64.161.96/32[gre]
Dec 10 05:05:59 vyos charon[12687]: 13[CFG] proposing traffic selectors for other:
Dec 10 05:05:59 vyos charon[12687]: 13[CFG] 116.90.86.181/32[gre]
Dec 10 05:05:59 vyos charon[12687]: 13[ENC] generating QUICK_MODE request 3607804314 [ HASH SA No KE ID ID ]
Dec 10 05:05:59 vyos charon[12687]: 13[NET] sending packet: from 100.64.161.96[4500] to 116.90.86.181[4500] (332 bytes)
Dec 10 05:05:59 vyos charon[12687]: 12[NET] received packet: from 116.90.86.181[4500] to 100.64.161.96[4500] (76 bytes)
Dec 10 05:05:59 vyos charon[12687]: 12[ENC] parsed INFORMATIONAL_V1 request 2361528290 [ HASH N(INVAL_ID) ]
Dec 10 05:05:59 vyos charon[12687]: 12[IKE] received INVALID_ID_INFORMATION error notify

show vpn debug

Status of IKE charon daemon (strongSwan 5.6.2, Linux 4.19.4-amd64-vyos, x86_64):
uptime: 18 hours, since Dec 09 11:49:39 2018
malloc: sbrk 2953216, mmap 0, used 1079040, free 1874176
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 63
loaded plugins: charon test-vectors ldap pkcs11 tpm aes rc2 sha2 sha1 md5 mgf1 rdrand random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent xcbc cmac hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default connmark stroke vici updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc xauth-generic xauth-eap xauth-pam xauth-noauth tnc-tnccs dhcp lookip error-notify certexpire led addrblock counters
Listening IP addresses:
100.64.161.96
Connections:
vpnprof-dmvpn-tun0: %any…%any IKEv1
vpnprof-dmvpn-tun0: local: [100.64.161.96] uses pre-shared key authentication
vpnprof-dmvpn-tun0: remote: uses pre-shared key authentication
dmvpn: child: dynamic[gre] === dynamic[gre] TUNNEL
Security Associations (1 up, 0 connecting):
vpnprof-dmvpn-tun0[554]: ESTABLISHED 70 seconds ago, 100.64.161.96[100.64.161.96]…116.90.86.181[116.90.86.181]
vpnprof-dmvpn-tun0[554]: IKEv1 SPIs: 1d80a49b252bba19_i* 4fee3d2118f59b23_r, rekeying in 57 minutes
vpnprof-dmvpn-tun0[554]: IKE proposal: AES_CBC_256/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024

In Hub Site:

Dec 10 05:11:38 vyos charon: 05[NET] sending packet: from 116.90.86.181[4500] to 115.60.62.155[1026] (76 bytes)
Dec 10 05:11:38 vyos charon: 06[NET] received packet: from 115.60.62.155[1026] to 116.90.86.181[4500] (332 bytes)
Dec 10 05:11:38 vyos charon: 06[ENC] parsed QUICK_MODE request 2409290503 [ HASH SA No KE ID ID ]
Dec 10 05:11:38 vyos charon: 06[CFG] looking for a child config for 116.90.86.181/32[gre] === 100.64.161.96/32[gre]
Dec 10 05:11:38 vyos charon: 06[CFG] proposing traffic selectors for us:
Dec 10 05:11:38 vyos charon: 06[CFG] 116.90.86.181/32[gre]
Dec 10 05:11:38 vyos charon: 06[CFG] proposing traffic selectors for other:
Dec 10 05:11:38 vyos charon: 06[CFG] 115.60.62.155/32[gre]
Dec 10 05:11:38 vyos charon: 06[IKE] no matching CHILD_SA config found

show vpn debug

Status of IKE charon daemon (strongSwan 5.6.2, Linux 4.19.4-amd64-vyos, x86_64):
uptime: 14 hours, since Dec 09 15:18:01 2018
malloc: sbrk 2973696, mmap 0, used 837248, free 2136448
worker threads: 10 of 16 idle, 5/0/1/0 working, job queue: 0/0/0/0, scheduled: 62
loaded plugins: charon test-vectors ldap pkcs11 tpm aesni aes rc2 sha2 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent xcbc cmac hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default connmark stroke vici updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc xauth-generic xauth-eap xauth-pam xauth-noauth tnc-tnccs dhcp lookip error-notify certexpire led addrblock counters
Listening IP addresses:
116.90.86.181
Connections:
vpnprof-dmvpn-tun0: %any…%any IKEv1
vpnprof-dmvpn-tun0: local: [116.90.86.181] uses pre-shared key authentication
vpnprof-dmvpn-tun0: remote: uses pre-shared key authentication
dmvpn: child: dynamic[gre] === dynamic[gre] TUNNEL
Security Associations (1 up, 1 connecting):
vpnprof-dmvpn-tun0[2]: CONNECTING, 116.90.86.181[%any]…192.168.200.1[%any]
vpnprof-dmvpn-tun0[2]: IKEv1 SPIs: ec31392f2e4f28e6_i* 0000000000000000_r
vpnprof-dmvpn-tun0[2]: Tasks queued: QUICK_MODE
vpnprof-dmvpn-tun0[2]: Tasks active: ISAKMP_VENDOR ISAKMP_CERT_PRE MAIN_MODE ISAKMP_CERT_POST ISAKMP_NATD
vpnprof-dmvpn-tun0[452]: ESTABLISHED 2 minutes ago, 116.90.86.181[116.90.86.181]…115.60.62.155[100.64.161.96]
vpnprof-dmvpn-tun0[452]: IKEv1 SPIs: 1d80a49b252bba19_i 4fee3d2118f59b23_r*, rekeying in 56 minutes
vpnprof-dmvpn-tun0[452]: IKE proposal: AES_CBC_256/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024

IP: 100.64.161.96/32 is Spock site pppoe interface IP address
IP: 115.60.62.155/32 is Spock site public IP address over nat
IP:116.90.86.181/32 is Hub site public static IP

may be help us help me fix that issue.

thanks
David

4

And I find log always have ike connect to 192.168.200.1 in HUB Vyos site. but I never set that IP address on Vyos

Dec 10 06:27:24 vyos charon[1797]: 05[NET] sending packet: from 116.90.86.181[500] to 192.168.200.1[500] (216 bytes)
Dec 10 06:28:39 vyos charon[1797]: 05[IKE] initiating Main Mode IKE_SA vpnprof-dmvpn-tun0[2] to 192.168.200.1
Dec 10 06:28:39 vyos charon[1797]: 05[NET] sending packet: from 116.90.86.181[500] to 192.168.200.1[500] (216 bytes)
Dec 10 06:28:43 vyos charon[1797]: 06[NET] sending packet: from 116.90.86.181[500] to 192.168.200.1[500] (216 bytes)
Dec 10 06:28:51 vyos charon[1797]: 13[NET] sending packet: from 116.90.86.181[500] to 192.168.200.1[500] (216 bytes)
Dec 10 06:29:04 vyos charon[1797]: 05[NET] sending packet: from 116.90.86.181[500] to 192.168.200.1[500] (216 bytes)
Dec 10 06:29:27 vyos charon[1797]: 06[NET] sending packet: from 116.90.86.181[500] to 192.168.200.1[500] (216 bytes)
Dec 10 06:30:09 vyos charon[1797]: 13[NET] sending packet: from 116.90.86.181[500] to 192.168.200.1[500] (216 bytes)