DMVPN setup - spokes can't ping each other

RedShift · October 2, 2025, 1:53pm

Hello all

Having a problem trying to setup a DMVPN network where the hub is a Cisco router and the spokes are VyOS routers.

The spokes can ping the hub’s DMVPN tunnel address
The hub can ping both spokes’ DMVPN tunnel address
Spokes can’t ping each other’s DMVPN tunnel address

The DMVPN network is 192.168.0.0/24:

192.168.0.1: cisco router hub address
192.168.0.2: vyos1 router spoke address
192.168.0.3: vyos2 router spoke address

ping 192.168.0.1 < - > 192.168.0.2: OK
ping 192.168.0.1 < - > 192.168.0.3: OK
ping 192.168.0.2 < - > 192.168.0.3: ERROR
ping 192.168.0.3 < - > 192.168.0.2: ERROR

Configuration and test results:

ciscorouter - DMVPN hub:
!
interface Loopback0
ip address 192.0.0.253 255.255.255.252
!
interface Tunnel99
ip address 192.168.0.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication aabbccdd
ip nhrp network-id 99
ip nhrp holdtime 300
ip nhrp redirect
no ip split-horizon
ip tcp adjust-mss 1360
ip ospf network broadcast
ip ospf priority 2
tunnel source Loopback0
tunnel mode gre multipoint
tunnel key 1
tunnel path-mtu-discovery
!
! This connects to vyos1:eth0
interface Vlan2001
ip address 192.0.0.1 255.255.255.252
!
! This connects to vyos2:eth0
interface Vlan2002
ip address 192.0.0.5 255.255.255.252
!

vyos1 - Spoke 1

set interfaces ethernet eth0 address ‘192.0.0.2/30’ # eth0 connects to ciscorouter:Vlan2001
set interfaces ethernet eth0 offload gro
set interfaces ethernet eth0 offload gso
set interfaces ethernet eth0 offload sg
set interfaces ethernet eth0 offload tso
set interfaces loopback lo
set interfaces tunnel tun99 address ‘192.168.0.2/24’
set interfaces tunnel tun99 enable-multicast
set interfaces tunnel tun99 encapsulation ‘gre’
set interfaces tunnel tun99 ip adjust-mss ‘1360’
set interfaces tunnel tun99 mtu ‘1400’
set interfaces tunnel tun99 parameters ip key ‘1’
set interfaces tunnel tun99 source-address ‘0.0.0.0’
set interfaces tunnel tun99 source-interface ‘eth0’
set protocols nhrp tunnel tun99 cisco-authentication ‘aabbccdd’
set protocols nhrp tunnel tun99 holding-time ‘300’
set protocols nhrp tunnel tun99 map 192.168.0.1 cisco
set protocols nhrp tunnel tun99 map 192.168.0.1 nbma-address ‘192.0.0.253’
set protocols nhrp tunnel tun99 map 192.168.0.1 register
set protocols nhrp tunnel tun99 multicast ‘nhs’
set protocols nhrp tunnel tun99 shortcut
set protocols static route 0.0.0.0/0 next-hop 192.0.0.1

vyos2 - Spoke 2

set interfaces ethernet eth0 address ‘192.0.0.6/30’ # eth0 connects to ciscrouter:Vlan2002
set interfaces ethernet eth0 offload gro
set interfaces ethernet eth0 offload gso
set interfaces ethernet eth0 offload sg
set interfaces ethernet eth0 offload tso
set interfaces loopback lo
set interfaces tunnel tun99 address ‘192.168.0.3/24’
set interfaces tunnel tun99 enable-multicast
set interfaces tunnel tun99 encapsulation ‘gre’
set interfaces tunnel tun99 ip adjust-mss ‘1360’
set interfaces tunnel tun99 mtu ‘1400’
set interfaces tunnel tun99 parameters ip key ‘1’
set interfaces tunnel tun99 source-address ‘0.0.0.0’
set interfaces tunnel tun99 source-interface ‘eth0’
set protocols nhrp tunnel tun99 cisco-authentication ‘aabbccdd’
set protocols nhrp tunnel tun99 holding-time ‘300’
set protocols nhrp tunnel tun99 map 192.168.0.1 cisco
set protocols nhrp tunnel tun99 map 192.168.0.1 nbma-address ‘192.0.0.253’
set protocols nhrp tunnel tun99 map 192.168.0.1 register
set protocols nhrp tunnel tun99 multicast ‘nhs’
set protocols nhrp tunnel tun99 shortcut
set protocols static route 0.0.0.0/0 next-hop 192.0.0.5

Test results from ciscorouter

[OK] Ping to vyos1 eth0
router#ping 192.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.0.0.2, timeout is 2 seconds:
!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
router#

[OK] Ping to vyos2 eth0
router#ping 192.0.0.6
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.0.0.6, timeout is 2 seconds:
!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
router#

[OK] Ping to vyos1 DMVPN tunnel
router#ping 192.168.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:
!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

[OK] Ping to vyos2 DMVPN tunnel

router#ping 192.168.0.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.3, timeout is 2 seconds:
!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
router#

Test results from vyos1

[OK] Ping from vyos1 to vyos2
vyos@vyos1:~$ ping 192.0.0.6 count 1
PING 192.0.0.6 (192.0.0.6) 56(84) bytes of data.
64 bytes from 192.0.0.6: icmp_seq=1 ttl=63 time=0.951 ms

— 192.0.0.6 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.951/0.951/0.951/0.000 ms

[OK] Ping from vyos1 to ciscorouter
vyos@vyos1:~$ ping 192.0.0.253 count 1
PING 192.0.0.253 (192.0.0.253) 56(84) bytes of data.
64 bytes from 192.0.0.253: icmp_seq=1 ttl=255 time=0.716 ms

— 192.0.0.253 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.716/0.716/0.716/0.000 ms

[OK] Ping from vyos1 to ciscorouter DMVPN tunnel
vyos@vyos1:~$ ping 192.168.0.1 count 1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=255 time=1.00 ms

— 192.168.0.1 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.003/1.003/1.003/0.000 ms

[ERROR] Ping from vyos1 to vyos2 DMVPN tunnel
vyos@vyos1:~$ ping 192.168.0.3 count 1
PING 192.168.0.3 (192.168.0.3) 56(84) bytes of data.
From 192.168.0.2 icmp_seq=1 Destination Host Unreachable

— 192.168.0.3 ping statistics —
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

Test results from vyos2

[OK] Ping from vyos2 to vyos1

vyos@vyos2:~$ ping 192.0.0.2 count 1
PING 192.0.0.2 (192.0.0.2) 56(84) bytes of data.
64 bytes from 192.0.0.2: icmp_seq=1 ttl=63 time=0.947 ms

— 192.0.0.2 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.947/0.947/0.947/0.000 ms

[OK] Ping from vyos2 to ciscorouter
vyos@vyos2:~$ ping 192.0.0.253 count 1
PING 192.0.0.253 (192.0.0.253) 56(84) bytes of data.
64 bytes from 192.0.0.253: icmp_seq=1 ttl=255 time=0.815 ms

— 192.0.0.253 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.815/0.815/0.815/0.000 ms

[OK] Ping from vyos1 to ciscorouter DMVPN tunnel
vyos@vyos2:~$ ping 192.168.0.1 count 1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=255 time=0.855 ms

— 192.168.0.1 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.855/0.855/0.855/0.000 ms

[ERROR] Ping from vyos2 to vyos1 DMVPN tunnel
vyos@vyos2:~$ ping 192.168.0.2 count 1
PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
From 192.168.0.3 icmp_seq=1 Destination Host Unreachable

— 192.168.0.2 ping statistics —
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

What else do I need to get spoke-to-spoke communcation to work?

L0crian · October 2, 2025, 2:23pm

I didn’t look at your full config, but you’re trying to do phase3, but you have phase2 configured for OSPF (network broadcast). Any protocol that uses next-hop-unchanged will be phase2.

That may not be your issue. If that’s not your issue, I’ll take a deeper look later if I have time

RedShift · October 2, 2025, 2:36pm

Yes I’m trying to configure for phase 2, the routing protocol in the production environment is OSPF (we also have no intention of changing this). I don’t think this is the issue though, the spokes should be able to ping each other regardless of a routing protocol being active or not (I tested this on a DMVPN network with all Cisco equipment and the spokes could ping each other like you would expect).

L0crian · October 2, 2025, 2:44pm

Gotcha, for phase2 you don’t use redirect/shortcut. The Cisco Hub will use incomplete CEF adjacencies to send the NBMA info to the spokes.

Like I said, I’ll take a deeper look when I find time, but just wanted to mention that for now.

L0crian · October 2, 2025, 3:21pm

What versions are you running for VyOS?

RedShift · October 2, 2025, 6:46pm

The version is VyOS 1.5-stream-2025-Q2

L0crian · October 2, 2025, 7:24pm

Your DMVPN is likely not actually up. The reason your pings work is because of the static defaults on the vyos routers. This is allowing them to reach 192.168.0.1 via 192.0.0.1 and 192.0.0.5. If you do show dmvpn on the Cisco router, you’ll likely see that the peer NBMA shows as unknown.

This is because of this line in your config:

set interfaces tunnel tun99 source-address ‘0.0.0.0’

In GRE, the source is not a match against interesting traffic, it’s the actual underlay IP that the GRE tunnel will use. So you’re saying that the source-address is actually 0.0.0.0 as an actual IP address; not a subnet.

Delete that line from the vyos routers. You’ll also want to adjust your static routes for your lab. When I lab DMVPN, I prefer to just put the hub(s) and spokes on a single layer 2 domain. This ensure reachability between devices on the underlay without needing to worry about routing.

RedShift · October 2, 2025, 7:34pm

I ran

set interfaces tunnel tun99 source-address ‘0.0.0.0’

However that did not resolve the issue. I also checked on the hub router and the DMVPN is definitely up:

router#show dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
        N - NATed, L - Local, X - No Socket
        T1 - Route Installed, T2 - Nexthop-override
        C - CTS Capable, I2 - Temporary
        # Ent --> Number of NHRP entries with same NBMA peer
        NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
        UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel99, IPv4 NHRP Details
Type:Hub, NHRP Peers:2,

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 192.0.0.2           192.168.0.2    UP    1d06h     D
     1 192.0.0.6           192.168.0.3    UP    1d06h     D

router#

L0crian · October 2, 2025, 7:37pm

Did you delete that source, or set it? You had it set in the config you listed before.

From the hub, try these:

ping 192.168.0.2 source 192.168.0.1
ping 192.168.0.3 source 192.168.0.1

If those work, then your DMVPN is indeed up (still delete that line for the source-address in there.

Add this line to your vyos configs:

set protocols static route 192.168.0.0/24 next-hop 192.168.0.1

Viacheslav · October 2, 2025, 10:46pm

Try to set /32 tunnel address + static route as in the documentation example.

RedShift · October 3, 2025, 9:42am

I tried setting the tunnel address to /32 with static route to the DMVPN network, however that did not work either. It’s also showing the route as inactive which I can’t figure out why:

vyos@vyos1:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

S>* 0.0.0.0/0 [1/0] via 192.0.0.1, eth0, weight 1, 03:01:10
C>* 172.16.0.0/24 is directly connected, eth1, 03:01:15
C>* 192.0.0.0/30 is directly connected, eth0, 03:01:13
S   192.168.0.0/24 [1/0] via 192.168.0.1 inactive, weight 1, 03:01:10
C>* 192.168.0.2/32 is directly connected, tun99, 03:01:12

The tunnel is up:

vyos@vyos1:~$ show int tunnel
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
tun99            192.168.0.2/32                    u/u
vyos@vyos1:~$

RedShift · October 3, 2025, 9:47am

Sorry yes I deleted that source, it is no longer present in the configuration:

delete interfaces tunnel tun99 source-address

From the hub I can ping the spokes with manual source address selection:

router#ping 192.168.0.2 source 192.168.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:
Packet sent with a source address of 192.168.0.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
router#

router#ping 192.168.0.3 source 192.168.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.3, timeout is 2 seconds:
Packet sent with a source address of 192.168.0.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
router#

L0crian · October 3, 2025, 1:59pm

The pings might be completing asymmetrically. Do this on the vyos routers and ensure the pings still complete:

vyos1:

delete protocols static route 0.0.0.0/0
set protocols static route 192.0.0.0/24 next-hop 192.0.0.1

vyos2:

delete protocols static route 0.0.0.0/0
set protocols static route 192.0.0.0/24 next-hop 192.0.0.5