DMVPN to Cisco router not come UP

dinukro · June 2, 2022, 6:47am

Hi all, the goal to to replace Cisco DMVPN working infrastrutture.
I configured the VyOS hub and Cisco Spoke router from documentation present at the link:

https://docs.vyos.io/en/equuleus/configuration/vpn/dmvpn.html

But the NHRP coming UP.

NOTE: I have PPPoE connection on all sites

This is the configuration (no IPsec configured):

HUB

set interfaces tunnel tun100 address ‘10.69.254.1/24’
set interfaces tunnel tun100 encapsulation ‘gre’
set interfaces tunnel tun100 source-address ‘xxx.xxx.xxx.121’
set interfaces tunnel tun100 multicast ‘enable’
set interfaces tunnel tun100 parameters ip key ‘1’

set protocols nhrp tunnel tun100 cisco-authentication ‘secret’
set protocols nhrp tunnel tun100 holding-time ‘300’
set protocols nhrp tunnel tun100 multicast ‘dynamic’
set protocols nhrp tunnel tun100 redirect
set protocols nhrp tunnel tun100 shortcut

Cisco Router

interface Tunnel10
! individual spoke tunnel IP must change
ip address 10.69.254.2 255.255.255.0
no ip redirects
ip nhrp authentication secret
ip nhrp map 10.69.254.1 62.97.39.121
ip nhrp map multicast xxx.xxx.39.121
ip nhrp network-id 1
ip nhrp holdtime 600
ip nhrp nhs 10.69.254.1
ip nhrp registration timeout 75
tunnel source dialer1
tunnel mode gre multipoint
tunnel key 1

RTRBR2#sh dmvpn

Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb

 1 xxx.xxx.xxx.121        10.69.254.1  NHRP 00:10:06     S

On the HUB i have used the command:

set interfaces tunnel tun100 source-interface pppoe0

but nothing change

Any one can help to solve the problem?
Thank you

e.khudiyev · June 2, 2022, 11:36am

Hi @dinukro ,

is this correct? I think you mean it doesn’t show UP… Are there any corresponding logs on Hub or Spoke device? What is the VyOS version you’re using in your tests? Does the connectivity between devices work? Also, try to remove source-interface on Hub site and specify the IP address instead.

dinukro · June 2, 2022, 12:07pm

Hi @e.khudiyev
the version is:

Version: VyOS 1.4-rolling-202205310217

This is the output from debug nhrp on Spoke side

Jun 2 11:53:49.433: NHRP: if_up: Tunnel10 proto ‘NHRP_IPv4’
Jun 2 11:53:49.433: NHRP: Registration with Tunnels Decap Module succeeded
Jun 2 11:53:49.433: NHRP: Adding all static maps to cache
Jun 2 11:53:49.433: NHRP: Adding multicast map entry to multicast list xxx.xxx.xxx.121
Jun 2 11:53:49.433: NHRP: Could not find AVL node for vrf:global(0x0)
Jun 2 11:53:49.433: NHRP: Inserted node for vrf global(0x0)
Jun 2 11:53:49.433: NHRP: Initialized cache radix head for vrf global(0x0)
Jun 2 11:53:49.433: NHRP: Adding Tunnel Endpoints (VPN: 10.69.254.1, NBMA: xxx.xxx.xxx.121)
Jun 2 11:53:49.433: NHRP: Successfully attached NHRP subblock for Tunnel Endpoints (VPN: 10.69.254.1, NBMA: xxx.xxx.xxx.121)
Jun 2 11:53:49.433: NHRP: No peer data updated in NHRP subblock for Tunnel Endpoints (VPN: 10.69.254.1, NBMA: xxx.xxx.xxx.121)
Jun 2 14:53:49: %PARSER-5-CFGLOG_LOGGEDCMD: User:fatbit logged command:no shutdown
Jun 2 11:53:50.433: NHRP: No SNMP node found to add requestID
Jun 2 11:53:50.433: NHRP: Multicast enabled for dst xxx.xxx.xxx.121
Jun 2 11:53:50.433: NHRP: NHS 10.69.254.1 Tunnel10 vrf 0 Cluster 0 Priority 0 Transitioned to ‘E’ from ’ ’
Jun 2 11:53:50.433: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:53:51.185: NHRP: Setting retrans delay to 2 for nhs dst 10.69.254.1
Jun 2 11:53:51.185: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 14:53:51: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel10, changed state to up
Jun 2 11:53:51.433: NHRP: if_up: Tunnel10 proto ‘NHRP_IPv4’
Jun 2 11:53:51.433: NHRP: Registration with Tunnels Decap Module succeeded
Jun 2 11:53:51.433: NHRP: Adding all static maps to cache
Jun 2 11:53:51.433: NHRP: Adding multicast map entry to multicast list xxx.xxx.xxx.121
Jun 2 11:53:51.433: NHRP: Adding Tunnel Endpoints (VPN: 10.69.254.1, NBMA: xxx.xxx.xxx.121)
Jun 2 11:53:51.433: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 10.69.254.1, NBMA: xxx.xxx.xxx.121)
Jun 2 11:53:51.433: NHRP: No peer data updated in NHRP subblock for Tunnel Endpoints (VPN: 10.69.254.1, NBMA: xxx.xxx.xxx.121)
Jun 2 11:53:51.433: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 10.69.254.1, NBMA: xxx.xxx.xxx.121)
Jun 2 11:53:51.433: NHRP: No SNMP node found to add requestID
Jun 2 11:53:51.433: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:53:51.433: NHRP: Resetting retransmit due to hold-timer for 10.69.254.1
Jun 2 14:53:51: %LINK-3-UPDOWN: Interface Tunnel10, changed state to up
Jun 2 11:53:53.041: NHRP: Setting retrans delay to 4 for nhs dst 10.69.254.1
Jun 2 11:53:53.041: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:53:56.197: NHRP: NHS-DOWN: 10.69.254.1
Jun 2 11:53:56.197: NHRP: Already pending Registration Request for NHS: 10.69.254.1
Jun 2 11:53:56.197: NHRP: NHS 10.69.254.1 Tunnel10 vrf 0 Cluster 0 Priority 0 Transitioned to ‘E’ from ‘E’
Jun 2 11:53:56.197: NHRP: Setting retrans delay to 8 for nhs dst 10.69.254.1
Jun 2 11:53:56.197: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:54:03.761: NHRP: Setting retrans delay to 16 for nhs dst 10.69.254.1
Jun 2 11:54:03.761: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:54:16.253: NHRP: Setting retrans delay to 32 for nhs dst 10.69.254.1
Jun 2 11:54:16.253: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:54:41.585: NHRP: Setting retrans delay to 64 for nhs dst 10.69.254.1
Jun 2 11:54:41.585: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:55:06.433: NHRP: No SNMP node found to add requestID
Jun 2 11:55:06.433: NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address
Jun 2 11:55:06.433: NHRP: Resetting retransmit due to hold-timer for 10.69.254.1

To take logs from VyOS please help me.
Thank you

e.khudiyev · June 2, 2022, 1:55pm

To check logs on VyOS you can use:

show log | match nhrp

From what I see, pay attention to the log containing:

NHRP: Supressing registration requests since self tunnel interface (Tunnel10) has invalid address

Also, the following command may help to check nhrp on Spoke side:

show ip nhrp nhs detail

dinukro · June 2, 2022, 5:24pm

Hi, thank you, this are the logs

Jun 02 14:04:50 opennhrp[10669]: Removing local 10.69.254.1/32 dev tun100 up
Jun 02 14:04:50 opennhrp[10669]: Removing local 10.69.254.255/32 alias 10.69.254.1 dev tun100 up
Jun 02 14:04:50 opennhrp[10669]: Filter code installed (19 opcodes)
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: config change, mtu=1476
Jun 02 14:04:50 opennhrp[10669]: ioctl(SIOCGETTUNNEL): No such device
Jun 02 14:04:50 opennhrp[10669]: ioctl(SIOCGIFFLAGS): No such device
Jun 02 14:04:50 opennhrp[10669]: Unknown NLmsg: 0x00000002, len 92
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: GRE configuration changed. Purged 0 peers.
Jun 02 14:04:50 opennhrp[10669]: Interface ‘tun100’ deleted
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: config change, mtu=0
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: GRE configuration changed. Purged 0 peers.
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: config change, mtu=1472
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: config change, mtu=1472
Jun 02 14:04:50 opennhrp[10669]: Adding local 10.69.254.1/32 dev tun100
Jun 02 14:04:50 opennhrp[10669]: Adding local 10.69.254.255/32 alias 10.69.254.1 dev tun100
Jun 02 14:04:50 opennhrp[10669]: Filter code installed (21 opcodes)
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: config change, mtu=1476
Jun 02 14:04:50 opennhrp[10669]: Interface tun100: configured UP, mtu=1476

And from Spoke

show ip nhrp nhs detail
IPv4 Registration Timer: 75 seconds

Legend: E=Expecting replies, R=Responding, W=Waiting
Tunnel10:
10.69.254.1 E priority = 0 cluster = 0 req-sent 0 req-failed 8 repl-recv 0

Pending Registration Requests:
Registration Request: Reqid 1354, Ret 64 NHS 10.69.254.1 expired (Tu10)

e.khudiyev · June 3, 2022, 1:26pm

Hi @dinukro , just checked on a few 1.4 rolling releases and seems that something is broken there. From my 1.4 setup, I found logs pointing to the following: Peer up script failed:…

On 1.3 everything worked and the peering came up

vyos@VyOS15-Hub:~$ show nhrp tunnel
Status: ok

Interface: tun100
Type: local
Protocol-Address: 10.69.254.255/32
Alias-Address: 10.69.254.1
Flags: up

Interface: tun100
Type: local
Protocol-Address: 10.69.254.1/32
Flags: up

Interface: tun100
Type: dynamic
Protocol-Address: 10.69.254.2/32
NBMA-Address: 100.65.0.121
Flags: up
Expires-In: 9:25

vIOS-Spoke#show dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
        N - NATed, L - Local, X - No Socket
        T1 - Route Installed, T2 - Nexthop-override
        C - CTS Capable
        # Ent --> Number of NHRP entries with same NBMA peer
        NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
        UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel10, IPv4 NHRP Details
Type:Spoke, NHRP Peers:1,

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 62.97.39.121        10.69.254.1    UP 00:04:27     S

I’ll try to re-check everything once more and create a bug report if it is confirmed.

dinukro · June 4, 2022, 8:43am

Hi, thank you for support, I can downgrade to 1.3; or better to reinstall; or I need to wait for next release 1,4 that fix the problem?

Thank you

e.khudiyev · June 4, 2022, 6:48pm

Do clean install of 1.3, it should work. You can report your issue with details on phabricator and check it once the issue is solved

dinukro · June 6, 2022, 11:44am

Hi @e.khudiyev, downgrade to 1.3.1 but nothing change

vyos@vyos:~$ show nhrp tunnel
Status: ok

Interface: tun100
Type: local
Protocol-Address: 10.69.254.255/32
Alias-Address: 10.69.254.1
Flags: up

Interface: tun100
Type: local
Protocol-Address: 10.69.254.1/32
Flags: up

Interface: tun100
Type: negative
Protocol-Address: 10.69.254.2/32
NBMA-Address: 79.113.63.216
Expires-In: 2:47

Jun 06 13:42:00 opennhrp[2001]: Received Registration Request from proto src 10.69.254.2 to 10.69.254.1
Jun 06 13:42:00 opennhrp[2001]: [10.69.254.2] Peer registration authorized
Jun 06 13:42:00 opennhrp[2001]: Removing negative 10.69.254.2/32 nbma 79.113.63.216 dev tun100 mtu 17912 expires_in 2:41
Jun 06 13:42:00 opennhrp[2001]: Adding dynamic 10.69.254.2/32 nbma 79.113.63.216 dev tun100 mtu 17912 expires_in 9:59
Jun 06 13:42:00 opennhrp[2001]: Sending Registration Reply from proto src 10.69.254.1 to 10.69.254.2 (1 bindings accepted, 0 rejected)
Jun 06 13:42:01 opennhrp[2001]: [10.69.254.2] Peer up script failed: exitstatus 1
Jun 06 13:42:02 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:42:10 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:42:26 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:42:28 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:42:32 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:42:33 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:42:35 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:42:39 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:42:40 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:42:45 opennhrp[2001]: Received Registration Request from proto src 10.69.254.3 to 10.69.254.1
Jun 06 13:42:45 opennhrp[2001]: [10.69.254.3] Peer registration authorized
Jun 06 13:42:45 opennhrp[2001]: Removing negative 10.69.254.3/32 nbma 79.113.60.240 dev tun100 mtu 17912 expires_in 1:56
Jun 06 13:42:45 opennhrp[2001]: Adding dynamic 10.69.254.3/32 nbma 79.113.60.240 dev tun100 mtu 17912 expires_in 9:59
Jun 06 13:42:45 opennhrp[2001]: Sending Registration Reply from proto src 10.69.254.1 to 10.69.254.3 (1 bindings accepted, 0 rejected)
Jun 06 13:42:45 opennhrp[2001]: [10.69.254.3] Peer up script failed: exitstatus 1
Jun 06 13:42:47 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:42:56 opennhrp[2001]: Received Registration Request from proto src 10.69.254.3 to 10.69.254.1
Jun 06 13:42:56 opennhrp[2001]: [10.69.254.3] Peer registration authorized
Jun 06 13:42:56 opennhrp[2001]: Removing negative 10.69.254.3/32 nbma 79.113.60.240 dev tun100 mtu 17912 expires_in 2:49
Jun 06 13:42:56 opennhrp[2001]: Adding dynamic 10.69.254.3/32 nbma 79.113.60.240 dev tun100 mtu 17912 expires_in 9:59
Jun 06 13:42:56 opennhrp[2001]: Sending Registration Reply from proto src 10.69.254.1 to 10.69.254.3 (1 bindings accepted, 0 rejected)
Jun 06 13:42:57 opennhrp[2001]: [10.69.254.3] Peer up script failed: exitstatus 1
Jun 06 13:43:03 opennhrp[2001]: Received Registration Request from proto src 10.69.254.2 to 10.69.254.1
Jun 06 13:43:04 opennhrp[2001]: [10.69.254.2] Peer registration authorized
Jun 06 13:43:04 opennhrp[2001]: Removing negative 10.69.254.2/32 nbma 79.113.63.216 dev tun100 mtu 17912 expires_in 1:57
Jun 06 13:43:04 opennhrp[2001]: Adding dynamic 10.69.254.2/32 nbma 79.113.63.216 dev tun100 mtu 17912 expires_in 9:59
Jun 06 13:43:04 opennhrp[2001]: Sending Registration Reply from proto src 10.69.254.1 to 10.69.254.2 (1 bindings accepted, 0 rejected)
Jun 06 13:43:04 opennhrp[2001]: [10.69.254.2] Peer up script failed: exitstatus 1
Jun 06 13:43:10 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:43:12 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:43:14 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:43:15 opennhrp[2001]: Received Registration Request from proto src 10.69.254.2 to 10.69.254.1
Jun 06 13:43:15 opennhrp[2001]: [10.69.254.2] Peer registration authorized
Jun 06 13:43:15 opennhrp[2001]: Removing negative 10.69.254.2/32 nbma 79.113.63.216 dev tun100 mtu 17912 expires_in 2:48
Jun 06 13:43:15 opennhrp[2001]: Adding dynamic 10.69.254.2/32 nbma 79.113.63.216 dev tun100 mtu 17912 expires_in 9:59
Jun 06 13:43:15 opennhrp[2001]: Sending Registration Reply from proto src 10.69.254.1 to 10.69.254.2 (1 bindings accepted, 0 rejected)
Jun 06 13:43:16 opennhrp[2001]: [10.69.254.2] Peer up script failed: exitstatus 1
Jun 06 13:43:16 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:43:16 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:43:20 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2
Jun 06 13:43:24 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.3
Jun 06 13:43:28 opennhrp[2001]: NL-ARP(tun100) who-has 10.69.254.2

e.khudiyev · June 6, 2022, 12:12pm

Could you please show the output for “show version command”?

dinukro · June 6, 2022, 12:21pm

Downgraded to 1.3.0 from 1.3.1, but nothing change

vyos@vyos:~$ show version

Version: VyOS 1.3.0
Release train: sagitta

Built by: [email protected]
Built on: Tue 18 Jan 2022 08:17 UTC
Build UUID: 6782b481-71c5-4518-a702-5095ad7164a9
Build commit ID: 52f75193f3afe7

Architecture: x86_64
Boot via: installed image
System type: bare metal

Hardware vendor: To be filled by O.E.M.
Hardware model: To be filled by O.E.M.
Hardware S/N: To be filled by O.E.M.
Hardware UUID: 03000200-0400-0500-0006-000700080009

Copyright: VyOS maintainers and contributors
vyos@vyos:~$

dinukro · June 6, 2022, 10:10pm

Hi @e.khudiyev, thank you for support, I build a EVE Lab, and the result is the same. The DMVPN is coming UP from Cisco to Cisco and VyOS to VyOS but never between VyOS to Cisco.

I read the documentation, and honestly I don’t understand what I miss.

VyOS Version

Version:          VyOS 1.3.1
Release train:    sagitta

Built by:         [email protected]
Built on:         Thu 14 Apr 2022 14:19 UTC
Build UUID:       4de5e5e1-16f2-4a5b-a142-4c435d503018
Build commit ID:  5fa31bdccc8bdf

IOL Version

IOS_To_VyOS#sh ver
Cisco IOS Software, Linux Software (I86BI_LINUX-ADVENTERPRISEK9-M), Version 15.7(3)M2, DEVELOPMENT TEST SOFTWARE

This is the configuration

Cisco DMVPN HUB

Cisco HUB


!
interface Tunnel0
 description mGRE - DMVPN Tunnel
 ip address 172.16.0.1 255.255.255.0
 no ip redirects
 ip nhrp authentication secret
 ip nhrp network-id 1
 tunnel source 1.0.0.1
 tunnel mode gre multipoint

Cisco Spoke

interface Tunnel0
 description Cisco_Spoke mGRE - DMVPN Tunnel
 ip address 172.16.0.3 255.255.255.0
 no ip redirects
 ip nhrp authentication secret
 ip nhrp map multicast 1.0.0.1
 ip nhrp map 172.16.0.1 1.0.0.1
 ip nhrp network-id 1
 ip nhrp nhs 172.16.0.1
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
end

VyOS Spoke

set interfaces ethernet eth0 address '4.0.0.1/24'
set interfaces tunnel tun0 address '172.16.0.4/24'
set interfaces tunnel tun0 encapsulation 'gre'
set interfaces tunnel tun0 multicast 'enable'
set interfaces tunnel tun0 parameters ip key '1'
set interfaces tunnel tun0 source-address '0.0.0.0'
set protocols nhrp tunnel tun0 cisco-authentication 'secret'
set protocols nhrp tunnel tun0 holding-time '300'
set protocols nhrp tunnel tun0 map 172.16.0.1/24 cisco
set protocols nhrp tunnel tun0 map 172.16.0.1/24 nbma-address '1.0.0.1'
set protocols nhrp tunnel tun0 map 172.16.0.1/24 register
set protocols nhrp tunnel tun0 multicast 'nhs'
set protocols nhrp tunnel tun0 redirect
set protocols nhrp tunnel tun0 shortcut

Result

Cisco HUB

Cisco_HUB#ping 3.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
Cisco_HUB#ping 4.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 4.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
Cisco_HUB#
Cisco_HUB#sh dmvpn 
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
        N - NATed, L - Local, X - No Socket
        T1 - Route Installed, T2 - Nexthop-override
        C - CTS Capable, I2 - Temporary
        # Ent --> Number of NHRP entries with same NBMA peer
        NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
        UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel0, IPv4 NHRP Details 
Type:Hub, NHRP Peers:1, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 3.0.0.1              172.16.0.3    UP 01:03:51     D
Cisco_HUB#ping 172.16.0.3 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.0.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
Cisco_HUB#

VyOS Spoke

vyos@Spoke-To-IOS:~$ sh nhrp tunnel 
Status: ok

Interface: tun0
Type: local
Protocol-Address: 172.16.0.255/32
Alias-Address: 172.16.0.4
Flags: up

Interface: tun0
Type: local
Protocol-Address: 172.16.0.4/32
Flags: up

Interface: tun0
Type: static
Protocol-Address: 172.16.0.1/24
NBMA-Address: 1.0.0.1
Flags: lower-up
vyos@Spoke-To-IOS:~$ show log | match nhrp
Jun 06 22:02:37 sudo[1601]:     root : PWD=/ ; USER=root ; COMMAND=/usr/bin/sh -c /usr/sbin/vyshim /usr/libexec/vyos/conf_mode/protocols_nhrp.py
Jun 06 22:02:37 vyos-configd[589]: Received message: {"type": "node", "data": "/usr/libexec/vyos/conf_mode/protocols_nhrp.py"}
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: OpenNHRP 0.14-20-g613277f starting
Jun 06 22:02:37 opennhrp[1610]: OpenNHRP 0.14-20-g613277f starting
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Adding static 172.16.0.1/24 nbma 1.0.0.1 dev tun0
Jun 06 22:02:37 opennhrp[1610]: Adding static 172.16.0.1/24 nbma 1.0.0.1 dev tun0
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface lo: configured UP, mtu=0
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface eth0: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface eth1: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface eth2: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface eth3: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface gre0: config change, mtu=1476
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface gretap0: config change, mtu=1462
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface erspan0: config change, mtu=1450
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface tun0: configured UP, mtu=1476
Jun 06 22:02:37 opennhrp[1610]: Interface lo: configured UP, mtu=0
Jun 06 22:02:37 opennhrp[1610]: Interface eth0: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: Interface eth1: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: Interface eth2: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: Interface eth3: configured UP, mtu=1500
Jun 06 22:02:37 opennhrp[1610]: Interface gre0: config change, mtu=1476
Jun 06 22:02:37 opennhrp[1610]: Interface gretap0: config change, mtu=1462
Jun 06 22:02:37 opennhrp[1610]: Interface erspan0: config change, mtu=1450
Jun 06 22:02:37 opennhrp[1610]: Interface tun0: configured UP, mtu=1476
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Interface tun0: GRE configuration changed. Purged 1 peers.
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Adding local 172.16.0.4/32 dev tun0
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Adding local 172.16.0.255/32 alias 172.16.0.4 dev tun0
Jun 06 22:02:37 opennhrp[1610]: opennhrp[1610]: Filter code installed (21 opcodes)
Jun 06 22:02:37 systemd[1]: opennhrp.service: Failed to parse PID from file /run/opennhrp/opennhrp.pid: Invalid argument
Jun 06 22:02:37 opennhrp[1610]: Interface tun0: GRE configuration changed. Purged 1 peers.
Jun 06 22:02:37 opennhrp[1610]: Adding local 172.16.0.4/32 dev tun0
Jun 06 22:02:37 opennhrp[1610]: Adding local 172.16.0.255/32 alias 172.16.0.4 dev tun0
Jun 06 22:02:37 opennhrp[1610]: Filter code installed (21 opcodes)
Jun 06 22:02:37 opennhrp[1620]: Interface tun0: config change, mtu=1476
Jun 06 22:02:42 opennhrp[1620]: [172.16.0.1] Peer up script: success
Jun 06 22:02:42 opennhrp[1620]: NL-ARP(tun0) 172.16.0.1 is-at 1.0.0.1
Jun 06 22:02:42 opennhrp[1620]: Sending Registration Request to 172.16.0.1 (my mtu=0)
Jun 06 22:02:42 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:02:47 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:02:52 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:02:57 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:03:02 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:03:07 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:03:12 opennhrp[1620]: Failed to register to 172.16.0.1: timeout (65535)
Jun 06 22:06:09 opennhrp[1620]: [172.16.0.1] Peer up script: success
Jun 06 22:06:09 opennhrp[1620]: NL-ARP(tun0) 172.16.0.1 is-at 1.0.0.1
Jun 06 22:06:09 opennhrp[1620]: Sending Registration Request to 172.16.0.1 (my mtu=0)
Jun 06 22:06:09 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:06:14 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:06:19 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:06:24 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:06:29 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:06:34 opennhrp[1620]: Sending packet 3, from: 172.16.0.4 (nbma 4.0.0.1), to: 172.16.0.1 (nbma 1.0.0.1)
Jun 06 22:06:39 opennhrp[1620]: Failed to register to 172.16.0.1: timeout (65535)

NOTE: enabled debug on Cisco HUB and never receive the registration from 4.0.0.1 (VyOS)

VyOS DMVPN HUB

VyOS HUB

set interfaces ethernet eth0 address '2.0.0.1/24'
set interfaces tunnel tun1 address '172.16.1.2/24'
set interfaces tunnel tun1 encapsulation 'gre'
set interfaces tunnel tun1 multicast 'enable'
set interfaces tunnel tun1 parameters ip key '1'
set interfaces tunnel tun1 source-address '2.0.0.1'
set protocols nhrp tunnel tun1 cisco-authentication 'secret'
set protocols nhrp tunnel tun1 holding-time '300'
set protocols nhrp tunnel tun1 multicast 'dynamic'
set protocols nhrp tunnel tun1 redirect
set protocols nhrp tunnel tun1 shortcut

VyOS Spoke

set interfaces ethernet eth0 address '5.0.0.1/24'
set interfaces tunnel tun1 address '172.16.1.5/24'
set interfaces tunnel tun1 encapsulation 'gre'
set interfaces tunnel tun1 multicast 'enable'
set interfaces tunnel tun1 parameters ip key '1'
set interfaces tunnel tun1 source-address '0.0.0.0'
set protocols nhrp tunnel tun1 cisco-authentication 'secret'
set protocols nhrp tunnel tun1 holding-time '300'
set protocols nhrp tunnel tun1 map 172.16.1.2/24 nbma-address '2.0.0.1'
set protocols nhrp tunnel tun1 map 172.16.1.2/24 register
set protocols nhrp tunnel tun1 multicast 'nhs'
set protocols nhrp tunnel tun1 redirect
set protocols nhrp tunnel tun1 shortcut

Cisco Spoke

interface Tunnel1
 description Cisco_Spoke mGRE - DMVPN Tunnel
 ip address 172.16.1.6 255.255.255.0
 no ip redirects
 ip nhrp authentication secret
 ip nhrp map multicast 2.0.0.1
 ip nhrp map 172.16.1.2 2.0.0.1
 ip nhrp network-id 1
 ip nhrp nhs 172.16.1.2
 tunnel source Ethernet0/0
 tunnel mode gre multipoint

Result

Cisco HUB

vyos@vyos:~$  ping 5.0.0.1
PING 5.0.0.1 (5.0.0.1) 56(84) bytes of data.
64 bytes from 5.0.0.1: icmp_seq=1 ttl=63 time=1.15 ms
64 bytes from 5.0.0.1: icmp_seq=2 ttl=63 time=1.53 ms
^C
--- 5.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.149/1.338/1.528/0.189 ms
vyos@vyos:~$  ping 6.0.0.1
PING 6.0.0.1 (6.0.0.1) 56(84) bytes of data.
64 bytes from 6.0.0.1: icmp_seq=1 ttl=254 time=1.15 ms
64 bytes from 6.0.0.1: icmp_seq=2 ttl=254 time=1.09 ms
^C
--- 6.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.085/1.116/1.148/0.031 ms
vyos@vyos:~$ 
vyos@vyos:~$ 
vyos@vyos:~$ 
vyos@vyos:~$ ping 172.16.1.5
PING 172.16.1.5 (172.16.1.5) 56(84) bytes of data.
64 bytes from 172.16.1.5: icmp_seq=1 ttl=64 time=1.04 ms
64 bytes from 172.16.1.5: icmp_seq=2 ttl=64 time=1.10 ms
^C
--- 172.16.1.5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.037/1.069/1.102/0.032 ms
vyos@vyos:~$ 
vyos@vyos:~$ 
vyos@vyos:~$ sh nhrp tunnel 
Status: ok

Interface: tun1
Type: local
Protocol-Address: 172.16.1.255/32
Alias-Address: 172.16.1.2
Flags: up

Interface: tun1
Type: local
Protocol-Address: 172.16.1.2/32
Flags: up

Interface: tun1
Type: dynamic
Protocol-Address: 172.16.1.5/32
NBMA-Address: 5.0.0.1
Flags: used up
Expires-In: 4:34

Cisco Spoke

IOS_To_VyOS#
*Jun  6 22:37:58.266: %SYS-5-CONFIG_I: Configured from console by console
IOS_To_VyOS#ping 2.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
IOS_To_VyOS#sh dm
IOS_To_VyOS#sh dmvpn 
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
        N - NATed, L - Local, X - No Socket
        T1 - Route Installed, T2 - Nexthop-override
        C - CTS Capable, I2 - Temporary
        # Ent --> Number of NHRP entries with same NBMA peer
        NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
        UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel1, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:1, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 2.0.0.1              172.16.1.2  NHRP 00:03:27     S

IOS_To_VyOS#
IOS_To_VyOS#show ip nhrp nhs detail
Legend: E=Expecting replies, R=Responding, W=Waiting
Tunnel1:
172.16.1.2   E priority = 0 cluster = 0  req-sent 9  req-failed 0  repl-recv 0 


Pending Registration Requests:
Registration Request: Reqid 1, Ret 64  NHS 172.16.1.2 expired (Tu1) 
IOS_To_VyOS#
IOS_To_VyOS# debug nhrp  
NHRP protocol debugging is on
No peer data updated in NHRP subblock for Tunnel Endpoints (VPN: 172.16.1.2, NBMA: 2.0.0.1)
*Jun  6 22:43:28.771: NHRP: Adding multicast map entry to multicast list 2.0.0.1
*Jun  6 22:43:29.773: NHRP: No SNMP node found to add requestID
*Jun  6 22:43:29.773: NHRP: Multicast enabled for dst 2.0.0.1
*Jun  6 22:43:29.773: NHRP: NHS 172.16.1.2 Tunnel1 vrf 0 Cluster 0 Priority 0 Transitioned to 'E' from ' ' 

*Jun  6 22:43:29.773: NHRP: Attempting to send packet through interface Tunnel1 via DEST  dst 172.16.1.2
*Jun  6 22:43:29.773: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 106
*Jun  6 22:43:29.773:       src: 172.16.1.6, dst: 172.16.1.2
*Jun  6 22:43:29.773: NHRP: Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 2.0.0.1
*Jun  6 22:43:29.773: NHRP: 130 bytes out Tunnel1 
*Jun  6 22:43:30.251: %SYS-5-CONFIG_I: Configured from console by console
*Jun  6 22:43:30.617: NHRP: Setting retrans delay to 2 for nhs  dst 172.16.1.2
*Jun  6 22:43:30.617: NHRP: Attempting to send packet through interface Tunnel1 via DEST  dst 172.16.1.2
*Jun  6 22:43:30.618: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 106
*Jun  6 22:43:30.618:       src: 172.16.1.6, dst: 172.16.1.2
*Jun  6 22:43:30.618: NHRP: Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 2.0.0.1
*Jun  6 22:43:30.618: NHRP: 130 bytes out Tunnel1 
*Jun  6 22:43:30.773: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
*Jun  6 22:43:30.773: NHRP: if_up: Tunnel1 proto 'NHRP_IPv4'
*Jun  6 22:43:30.773: NHRP: Registration with Tunnels Decap Module succeeded
*Jun  6 22:43:30.773: NHRP: Adding all static maps to cache
*Jun  6 22:43:30.773: NHRP: Adding Tunnel Endpoints (VPN: 172.16.1.2, NBMA: 2.0.0.1)
*Jun  6 22:43:30.773: NHRP: NHRP subblock already exists for Tunnel Endpoints (VPN: 172.16.1.2, NBMA: 2.0.0.1)
*Jun  6 22:43:30.773: NHRP: No peer data updated in NHRP subblock for Tunnel Endpoints (VPN: 172.16.1.2, NBMA: 2.0.0.1)
*Jun  6 22:43:30.773: NHRP: Cache already has a subblock node attached for Tunnel Endpoints (VPN: 172.16.1.2, NBMA: 2.0.0.1)
*Jun  6 22:43:30.773: NHRP: Adding multicast map entry to multicast list 2.0.0.1
*Jun  6 22:43:30.773: NHRP: No SNMP node found to add requestID
*Jun  6 22:43:30.773: NHRP: Attempting to send packet through interface Tunnel1 via DEST  dst 172.16.1.2
*Jun  6 22:43:30.773: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 106
*Jun  6 22:43:30.773:       src: 172.16.1.6, dst: 172.16.1.2
*Jun  6 22:43:30.773: NHRP: Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 2.0.0.1
*Jun  6 22:43:30.773: NHRP: 130 bytes out Tunnel1 
*Jun  6 22:43:30.773: NHRP: Resetting retransmit due to hold-timer for 172.16.1.2
*Jun  6 22:43:30.773: %LINK-3-UPDOWN: Interface Tunnel1, changed state to up
*Jun  6 22:43:32.309: NHRP: Setting retrans delay to 4 for nhs  dst 172.16.1.2
*Jun  6 22:43:32.309: NHRP: Attempting to send packet through interface Tunnel1 via DEST  dst 172.16.1.2
*Jun  6 22:43:32.309: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 106
*Jun  6 22:43:32.309:       src: 172.16.1.6, dst: 172.16.1.2
*Jun  6 22:43:32.309: NHRP: Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 2.0.0.1
*Jun  6 22:43:32.309: NHRP: 130 bytes out Tunnel1 
*Jun  6 22:43:36.155: NHRP: NHS-DOWN: 172.16.1.2
*Jun  6 22:43:36.155: NHRP: Already pending Registration Request for NHS: 172.16.1.2
*Jun  6 22:43:36.155: NHRP: NHS 172.16.1.2 Tunnel1 vrf 0 Cluster 0 Priority 0 Transitioned to 'E' from 'E' 

*Jun  6 22:43:36.155: NHRP: Setting retrans delay to 8 for nhs  dst 172.16.1.2
*Jun  6 22:43:36.155: NHRP: Attempting to send packet through interface Tunnel1 via DEST  dst 172.16.1.2
*Jun  6 22:43:36.155: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 106
*Jun  6 22:43:36.155:       src: 172.16.1.6, dst: 172.16.1.2
*Jun  6 22:43:36.155: NHRP: 
IOS_To_VyOS#Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 2.0.0.1
*Jun  6 22:43:36.155: NHRP: 130 bytes out Tunnel1 
IOS_To_VyOS#
*Jun  6 22:43:44.034: NHRP: Setting retrans delay to 16 for nhs  dst 172.16.1.2
*Jun  6 22:43:44.034: NHRP: Attempting to send packet through interface Tunnel1 via DEST  dst 172.16.1.2
*Jun  6 22:43:44.034: NHRP: Send Registration Request via Tunnel1 vrf global(0x0), packet size: 106
*Jun  6 22:43:44.034:       src: 172.16.1.6, dst: 172.16.1.2
*Jun  6 22:43:44.034: NHRP: Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 2.0.0.1
*Jun  6 22:43:44.034: NHRP: 130 bytes out Tunnel1 
IOS_To_VyOS#

dinukro · June 8, 2022, 7:21am

Hi, I tested more release and with 1.1.8 and 1.2.8 the tunnel between Cisco IOS and VyOS is UP

VyOS Version

vyos@vyos:~$  sh ver
Version:      VyOS 1.1.8
Description:  VyOS 1.1.8 (helium)
Copyright:    2017 VyOS maintainers and contributors
Built by:     [email protected]
Built on:     Sat Nov 11 13:44:36 UTC 2017
Build ID:     1711111344-b483efc

Tunnel config

set interfaces tunnel tun0 address '172.16.1.2/24'
set interfaces tunnel tun0 encapsulation 'gre'
set interfaces tunnel tun0 local-ip '2.0.0.1'
set interfaces tunnel tun0 multicast 'enable'
set interfaces tunnel tun0 parameters ip key '1'
set protocols nhrp tunnel tun0 cisco-authentication 'cisco'
set protocols nhrp tunnel tun0 holding-time '300'
set protocols nhrp tunnel tun0 multicast 'dynamic'
set protocols nhrp tunnel tun0 'redirect'
set protocols nhrp tunnel tun0 'shortcut'

IOS Version

Cisco IOS Software, Linux Software (I86BI_LINUX-ADVENTERPRISEK9-M), Version 15.7(3)M2, DEVELOPMENT TEST SOFTWARE

Tunnel Configuration

interface Tunnel1
 description Cisco_Spoke mGRE - DMVPN Tunnel
 ip address 172.16.1.6 255.255.255.0
 no ip redirects
 ip nhrp authentication cisco
 ip nhrp map multicast 2.0.0.1
 ip nhrp map 172.16.1.2 2.0.0.1
 ip nhrp network-id 1
 ip nhrp nhs 172.16.1.2
 ip nhrp registration timeout 75
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 1

Test

Cisco Router

IOS_To_VyOS#sh dmvpn 
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
        N - NATed, L - Local, X - No Socket
        T1 - Route Installed, T2 - Nexthop-override
        C - CTS Capable, I2 - Temporary
        # Ent --> Number of NHRP entries with same NBMA peer
        NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
        UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel1, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:1, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 2.0.0.1              172.16.1.2    UP 08:16:40     S

IOS_To_VyOS#ping 172.16.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/5/6 ms

VyOS

vyos@vyos:~$ sh nhrp tunnel 
Status: ok

Interface: tun0
Type: local
Protocol-Address: 172.16.1.255/32
Alias-Address: 172.16.1.2
Flags: up

Interface: tun0
Type: local
Protocol-Address: 172.16.1.2/32
Flags: up

Interface: tun0
Type: dynamic
Protocol-Address: 172.16.1.5/32
NBMA-Address: 5.0.0.1
NBMA-NAT-OA-Address: 192.168.1.1
Flags: up
Expires-In: 4:12

system · June 10, 2022, 7:22am

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.