Unexpected NHRP traffic flow.

I’ve been labbing a DMVPN setup with a Cisco Hub (on a DSL line) and two VyOS Spokes (both connected at my home lab same Gig VLAN). I’m running BGP between the routers. Unexpectedly, all spoke-to-spoke traffic appears to be flowing through the hub.

Ping times from Spokes to Hub is approximately 75ms.
Ping times from Spoke to Spoke (public IP’s) are sub millisecond (same Gigabit LAN)
Ping times from Spoke to Spoke (internal encrypted LAN) are approximately 75ms.

I can watch the flows on the Cisco Hub and definitely see the spoke-to-spoke traffic showing up on the Hub TUN interface.

HUB 10.0.0.1
Spoke 1 10.0.0.2
Spoke 2 10.0.0.4

I keep seeing this in the Spoke 1 Log:

Apr 3 08:46:19 Spoke1 pluto[4187]: “10.0.0.2-to-10.0.0.4”: deleting connection
Apr 3 08:46:19 Spoke1 pluto[4187]: two interfaces match “10.0.0.2-to-10.0.0.4” (eth0, eth0)
Apr 3 08:46:19 Spoke1 pluto[4187]: added connection description “10.0.0.2-to-10.0.0.4”
Apr 3 08:46:19 Spoke1 opennhrp[28685]: [10.0.0.4] Peer up script failed: exitstatus 1

Spoke 2 shows no such entries.

It would appear that the tunnels build:

Spoke1# run sho nhrp tun
Status: ok

Interface: tun0
Type: local
Protocol-Address: 10.0.0.255/32
Alias-Address: 10.0.0.2
Flags: up

Interface: tun0
Type: local
Protocol-Address: 10.0.0.2/32
Flags: up

Interface: tun0
Type: negative
Protocol-Address: 10.0.0.4/32
NBMA-Address:
Flags: used
Expires-In: 0:25

Interface: tun0
Type: static
Protocol-Address: 10.0.0.1/24
NBMA-Address:
Flags: used up

Spoke2 run sho nhrp tun

Status: ok

Interface: tun0
Type: local
Protocol-Address: 10.0.0.255/32
Alias-Address: 10.0.0.4
Flags: up

Interface: tun0
Type: local
Protocol-Address: 10.0.0.4/32
Flags: up

Interface: tun0
Type: cached
Protocol-Address: 10.0.0.2/32
NBMA-Address:
Flags: used up
Expires-In: 64:52

Interface: tun0
Type: static
Protocol-Address: 10.0.0.1/24
NBMA-Address:
Flags: used up

Config bits:

Spoke#sho int tun tun0
address
encapsulation gre
local-ip
multicast enable
parameters {
ip {
key 0
}
}

Spoke# sho protocols nhrp
tunnel tun0 {
cisco-authentication
map 10.0.0.1/24 {
cisco
nbma-address
register
}
multicast nhs
redirect
shortcut
}

Cisco HUB:

interface Tunnel0
ip address 10.0.0.1 255.255.255.0
no ip redirects
ip flow ingress
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp holdtime 600
ip nhrp registration timeout 75
ip nhrp shortcut
load-interval 30
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 0
tunnel protection ipsec profile DMVPN
end

Without “shortcut” on the spokes, the ping times from spoke-to-spoke internal LANs doubles (~75ms to ~150ms).

Am I just misunderstanding DMPVN or is something really not working right here?

As both spokes are on the same LAN, are and equal? That might mess up setting up vpn inbetween them.
Even more when they’re both behind same NAT device

Nah…Every device has it’s own unique IP address, all are public, none are behind NATs.

A power outage at James Corp HQ forced a reboot of the spokes. When they came back…They peered right up. Spoke-to-Spoke traffic is working as expected.

The grumpy Sys-Admin mantra “Shut up and Reboot” appears to have fixed this issue. Not really what I expected out of a Linux based distro like VyOS, but whatever.