Hi all,
i have found a problem with DMVPN: when both spokes are behind the NAT and one of the spokes is a Cisco router, VyOS hasn’t learnt the correct NBMA-Address for the Cisco router
my topology first
HUB is connected to the Internet through eth0 with a fixed public IP 207.148.116.a
Spoke1 is connected to a 1:1-NAT firewall through eth0 with the inside IP 10.65.138.33, and a fixed public IP 8.222.135.b NATed by the firewall.
Spoke2 is connected to the ISP through GigabitEthernet0/0/0 with an inside DHCP IP of 100.85.31.228 in this case. The public IP 103.252.202.c is one of the IPs in the ISP’s CGNAT pool.
DMVPN tunnel interface
- HUB
tun645170: 10.254.0.1/29 - Spoke1
tun645170: 10.254.0.2/29 - Spoke2
tun645170: 10.254.0.6/29
Platform and version
HUB is running VyOS with version VyOS 1.5-stream-2025-Q1
Spoke1 is running VyOS with version VyOS 1.4.0
Spoke2 is running Cisco IOS XE Software, Version 16.09.02
Phenomenon
Wait for the DMVPN and IPSEC to be established.
HUB ←→ Spoke1 can ping each other successfully.
HUB ←→ Spoke2 can ping each other successfully.
Spoke1 and Spoke2 CANNOT ping each other.
checked the NHRP table on each device, found that in Spoke1’s NHRP table, NBMA-Address of Spoke2 was not correct (it’s Spoke1 itself)
- NHRP table on HUB (correct)
xxxxxx@hub:~$ show nhrp tunnel
Status: ok
Interface Type Protocol-Address Alias-Address Flags NBMA-Address NBMA-NAT-OA-Address Expires-In
----------- ------- ------------------ --------------- ------- --------------- --------------------- ------------
tun645170 local 10.254.0.7/32 10.254.0.1 up
tun645170 local 10.254.0.1/32 up
tun645170 local 10.254.0.7/32 10.254.0.1 up
tun645170 local 10.254.0.1/32 up
tun645170 dynamic 10.254.0.6/32 used up 103.252.202.c 100.85.31.228 6:46
tun645170 dynamic 10.254.0.2/32 up 8.222.135.b 10.65.138.33 115:58
xxxxxx@hub:~$
- NHRP table on Spoke1 (not correct)
xxxxxx@spoke1:~$ show nhrp tunnel
Status: ok
Interface Type Protocol-Address Alias-Address Flags NBMA-Address NBMA-NAT-OA-Address Expires-In
----------- ------ ------------------ --------------- ------- -------------- --------------------- ------------
tun645170 local 10.254.0.7/32 10.254.0.2 up
tun645170 local 10.254.0.2/32 up
tun645170 cached 10.254.0.6/32 up 8.222.135.b 100.85.31.228 7:25
tun645170 static 10.254.0.1/29 used up 207.148.116.a
xxxxxx@spoke1:~$
Here’s the problem: the NBMA-Address of 10.254.0.6/32 should be the same as the HUB’s 103.252.202.c, but actually, it is the NATed public IP address of itself
- NHRP table on Spoke2 (correct)
spoke2#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
T1 - Route Installed, T2 - Nexthop-override
C - CTS Capable, I2 - Temporary
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel645170 is up/up, Addr. is 10.254.0.6, VRF ""
Tunnel Src./Dest. addr: 100.85.31.228/Multipoint, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect "ipsec-transport-aes256"
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 NHS:
10.254.0.1 RE NBMA Address: 207.148.116.a priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 5
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 207.148.116.a 10.254.0.1 UP 02:30:04 S 10.254.0.1/32
1 8.222.135.b 10.254.0.2 UP 17:37:25 DN 10.254.0.2/32
Claimed Addr. 10.65.138.33
1 100.85.31.228 10.254.0.6 UP 02:30:19 DLX 10.254.0.6/32
Crypto Session Details:
--------------------------------------------------------------------------------
Interface: Tunnel645170
Session: [0x7F782B37E0]
Session ID: 76
IKEv2 SA: local 100.85.31.228/4500 remote 207.148.116.a /4500 Active
Capabilities:DN connid:8 lifetime:02:53:47
Crypto Session Status: UP-ACTIVE
fvrf: (none), Phase1_id: 207.148.116.a
IPSEC FLOW: permit 47 host 100.85.31.228 host 207.148.116.a
Active SAs: 2, origin: crypto map
Inbound: #pkts dec'ed 20366 drop 0 life (KB/Sec) 4607807/962
Outbound: #pkts enc'ed 10231 drop 0 life (KB/Sec) 4607870/962
Outbound SPI : 0xC5BCDA0F, transform : esp-256-aes esp-sha-hmac
Socket State: Open
Interface: Tunnel645170
Session: [0x7F782B3AE0]
Session ID: 88
IKEv2 SA: local 100.85.31.228/4500 remote 8.222.135.b /4500 Active
Capabilities:DN connid:9 lifetime:06:12:13
Crypto Session Status: UP-ACTIVE
fvrf: (none), Phase1_id: 10.65.138.33
IPSEC FLOW: permit 47 host 100.85.31.228 host 8.222.135.b
Active SAs: 2, origin: crypto map
Inbound: #pkts dec'ed 0 drop 0 life (KB/Sec) 4608000/1126
Outbound: #pkts enc'ed 77 drop 0 life (KB/Sec) 4607999/1126
Outbound SPI : 0xCA1C038A, transform : esp-256-aes esp-sha-hmac
Socket State: Open
Pending DMVPN Sessions:
spoke2#
- vpn ipsec table on HUB (correct)
xxxxxxx@hub:~$ show vpn ipsec sa
Connection State Uptime Bytes In/Out Packets In/Out Remote address Remote ID Proposal
------------ ------- -------- -------------- ---------------- ---------------- --------------------- ----------------------------------
dmvpn up 35m41s 18K/59K 240/519 8.222.135.b 10.65.138.33 AES_CBC_256/HMAC_SHA1_96/MODP_1024
dmvpn up 49s 540B/1K 5/17 103.252.202.c gateway.sg.home.ipsec AES_CBC_256/HMAC_SHA1_96/MODP_1024
xxxxxxx@hub:~$
- vpn ipsec table on Spoke1 (not correct)
xxxxxx@spoke1:~$ show vpn ipsec sa
Connection State Uptime Bytes In/Out Packets In/Out Remote address Remote ID Proposal
------------ ------- -------- -------------- ---------------- ---------------- --------------------- ----------------------------------
dmvpn up 1m10s 0B/0B 0/0 103.252.202.c gateway.sg.home.ipsec AES_CBC_256/HMAC_SHA1_96/MODP_1024
dmvpn up 37m27s 0B/2M 0/21K 8.222.135.b 10.65.138.33 AES_CBC_256/HMAC_SHA1_96/MODP_1024
dmvpn up 37m27s 2M/0B 21K/0 8.222.135.b 10.65.138.33 AES_CBC_256/HMAC_SHA1_96/MODP_1024
dmvpn up 38m3s 63K/19K 553/256 207.148.116.a 207.148.116.a AES_CBC_256/HMAC_SHA1_96/MODP_1024
xxxxxx@spoke1:~$
Here’s another problem: because the DMVPN did not obtain the correct NBMA-Address of Spoke2 and used its own NATed IP address instead, IPSec ended up establishing the connection with Spoke1 itself.
- crypto session on Spoke2 (correct)
spoke2# show crypto session
Crypto session current status
Interface: Tunnel645170
Profile: ikev2-nat-any
Session status: UP-ACTIVE
Peer: 8.222.135.b port 4500
Session ID: 88
IKEv2 SA: local 100.85.31.228/4500 remote 8.222.135.b /4500 Active
IPSEC FLOW: permit 47 host 100.85.31.228 host 8.222.135.b
Active SAs: 2, origin: crypto map
Interface: Tunnel645170
Profile: ikev2-nat-any
Session status: UP-ACTIVE
Peer: 207.148.116.a port 4500
Session ID: 76
IKEv2 SA: local 100.85.31.228/4500 remote 207.148.116.a /4500 Active
IPSEC FLOW: permit 47 host 100.85.31.228 host 207.148.116.a
Active SAs: 2, origin: crypto map
spoke2#
Configurations
- HUB
interfaces {
ethernet eth0 {
address dhcp
}
tunnel tun645170 {
address 10.254.0.1/29
enable-multicast
encapsulation gre
mtu 1472
parameters {
ip {
key 645170
}
}
source-interface eth0
}
}
protocols {
nhrp {
tunnel tun645170 {
multicast dynamic
redirect
shortcut
}
}
}
vpn {
ipsec {
esp-group transport-aes256-sha1 {
lifetime 3600
mode transport
pfs dh-group2
proposal 1 {
encryption aes256
hash sha1
}
}
ike-group ikev2-aes256-sha1 {
close-action none
dead-peer-detection {
action clear
interval 10
timeout 50
}
ikev2-reauth
key-exchange ikev2
lifetime 28800
proposal 1 {
dh-group 2
encryption aes256
hash sha1
}
}
interface eth0
log {
level 1
subsystem mgr
subsystem ike
subsystem chd
subsystem knl
subsystem net
subsystem dmn
}
options {
disable-route-autoinstall
}
profile sg-dmvpn {
authentication {
mode pre-shared-secret
pre-shared-secret xxxxxxxx
}
bind {
tunnel tun645170
}
esp-group transport-aes256-sha1
ike-group ikev2-aes256-sha1
}
}
}
- Spoke1
interfaces {
ethernet eth0 {
address dhcp
description [WAN]8.222.135.b
hw-id 00:16:3e:10:17:57
offload {
gro
gso
}
}
tunnel tun645170 {
address 10.254.0.2/29
enable-multicast
encapsulation gre
mtu 1472
parameters {
ip {
key 645170
}
}
source-interface eth0
}
}
protocols {
nhrp {
tunnel tun645170 {
map 10.254.0.1/29 {
nbma-address 207.148.116.a
register
}
multicast nhs
redirect
shortcut
}
}
}
vpn {
ipsec {
esp-group transport-aes256-sha1 {
lifetime 3600
mode transport
pfs dh-group2
proposal 1 {
encryption aes256
hash sha1
}
}
ike-group ikev2-aes256-sha1 {
close-action none
dead-peer-detection {
action clear
interval 10
}
key-exchange ikev2
lifetime 28800
proposal 1 {
dh-group 2
encryption aes256
hash sha1
}
}
interface eth0
log {
level 1
subsystem mgr
subsystem ike
subsystem chd
subsystem knl
subsystem net
subsystem dmn
}
options {
disable-route-autoinstall
}
profile sg-dmvpn {
authentication {
mode pre-shared-secret
pre-shared-secret xxxxxxxx
}
bind {
tunnel tun645170
}
esp-group transport-aes256-sha1
ike-group ikev2-aes256-sha1
}
}
}
- Spoke2
Current configuration : 12635 bytes
!
! Last configuration change at 18:58:50 SIN Sat Nov 8 2025 by wolf
! NVRAM config last updated at 18:24:21 SIN Thu Nov 6 2025 by wolf
!
version 16.9
!
!
crypto ikev2 proposal AES256-SHA1-MODP1024
encryption aes-cbc-256
integrity sha1
group 2
crypto ikev2 proposal AES256-SHA256-MODP1024
encryption aes-cbc-256
integrity sha256
group 2
!
crypto ikev2 policy AES256-SHA1-MODP1024
proposal AES256-SHA1-MODP1024
crypto ikev2 policy sg-dmvpn
proposal AES256-SHA1-MODP1024
proposal AES256-SHA256-MODP1024
!
crypto ikev2 keyring sg-dmvpn
peer hub-sg-vultr
address 207.148.116.a
pre-shared-key xxxxxxxx
!
peer spoke-sg-ali
address 8.222.135.b
pre-shared-key xxxxxxxx
!
!
!
crypto ikev2 profile ikev2-nat-any
match identity remote any
identity local fqdn gateway.sg.home.ipsec
authentication remote pre-share
authentication local pre-share
keyring local sg-dmvpn
lifetime 28800
no lifetime certificate
dpd 10 3 periodic
nat keepalive 5
nat force-encap
!
crypto ipsec transform-set TRANSPORT-ESP-AES256-SHA1 esp-aes 256 esp-sha-hmac
mode transport
!
crypto ipsec df-bit clear
!
!
crypto ipsec profile ipsec-transport-aes256
set transform-set TRANSPORT-ESP-AES256-SHA1
set pfs group2
!
!
interface Tunnel645170
ip address 10.254.0.6 255.255.255.248
no ip redirects
ip nhrp network-id 645170
ip nhrp nhs 10.254.0.1 nbma 207.148.116.a multicast
ip nhrp redirect
ip ospf network broadcast
tunnel source GigabitEthernet0/0/0
tunnel mode gre multipoint
tunnel key 645170
tunnel protection ipsec profile ipsec-transport-aes256 ikev2-profile ikev2-nat-any
!
interface GigabitEthernet0/0/0
description WAN
ip dhcp client default-router distance 10
ip address dhcp
ip nat outside
negotiation auto
!
At the end
I’m not sure whether this issue is a bug or a misconfiguration on my part. It has been bothering me for several days. If anyone has experienced something similar, I would really appreciate your guidance.
Feel free to leave any comment; it will be helpful to me. Kindly let me know if you need something!
Thank you!
Regards,
Nicolas