VyOS Encrypted VXLAN Tunneling

omidit · October 12, 2021, 1:39pm

I used the following link to set up VXLAN Tunneling via an IPSEC/GRE Tunnel.
link:

https://www.datai.net/article/vyos-encrypted-vxlan-tunneling/

The configuration link is specifically for v1.2.0*. But I changed it to use VyOS 1.4-rolling-202109060217 version for both routers I have in my scenario.

vyos@vyos:~$ show version

Version:          VyOS 1.4-rolling-202109060217
Release Train:    sagitta

Built by:         [email protected]
Built on:         Mon 06 Sep 2021 02:17 UTC
Build UUID:       46886a29-3f26-4e68-93a5-f8f93b5c47d4
Build Commit ID:  8b8a3ff535b347

Architecture:     x86_64
Boot via:         installed image
System type:      VMware guest

Hardware vendor:  VMware, Inc.
Hardware model:   VMware Virtual Platform
Hardware S/N:     VMware-56 4d e9 ca 9b f2 38 d8-82 31 97 51 64 f7 94 ed
Hardware UUID:    564de9ca-9bf2-38d8-8231-975164f794ed

Copyright:        VyOS maintainers and contributors

Although I followed the same steps as the referred link, I found that some commands do not exist anymore in this version (VyOS 1.4-rolling), so changed them.

My Scenario:

ipsec_scenario.drawio

The following configurations are used for router-vyos-1 and router-vyos-2.

Router R1 and R2

Phase 1 - IPSEC IKEv2

set vpn ipsec ike-group aes256-sha256 dead-peer-detection action 'restart'
set vpn ipsec ike-group aes256-sha256 dead-peer-detection interval '30'
set vpn ipsec ike-group aes256-sha256 dead-peer-detection timeout '120'
set vpn ipsec ike-group aes256-sha256 ikev2-reauth 'no'
set vpn ipsec ike-group aes256-sha256 key-exchange 'ikev2'
set vpn ipsec ike-group aes256-sha256 lifetime '3600'
set vpn ipsec ike-group aes256-sha256 proposal 1 dh-group '19'
set vpn ipsec ike-group aes256-sha256 proposal 1 encryption 'aes256'
set vpn ipsec ike-group aes256-sha256 proposal 1 hash 'sha256'

Phase 2 - ESP Tunnel

set vpn ipsec esp-group aes256-sha256 compression 'disable'
set vpn ipsec esp-group aes256-sha256 lifetime '28800'
set vpn ipsec esp-group aes256-sha256 mode 'tunnel'
set vpn ipsec esp-group aes256-sha256 pfs 'dh-group19'
set vpn ipsec esp-group aes256-sha256 proposal 1 encryption 'aes256'
set vpn ipsec esp-group aes256-sha256 proposal 1 hash 'sha256'

Router-1

IPSec

set vpn ipsec site-to-site peer 172.16.1.2 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 172.16.1.2 authentication pre-shared-secret '3mqBSixtMiDy4ngWUKt0uScGH9f2vPoy'
set vpn ipsec site-to-site peer 172.16.1.2 connection-type 'initiate'
set vpn ipsec site-to-site peer 172.16.1.2 ike-group 'aes256-sha256'
set vpn ipsec site-to-site peer 172.16.1.2 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 172.16.1.2 local-address '172.16.1.1'
set vpn ipsec site-to-site peer 172.16.1.2 tunnel 1 esp-group 'aes256-sha256'
set vpn ipsec site-to-site peer 172.16.1.2 tunnel 1 local prefix '10.10.0.1/32'
set vpn ipsec site-to-site peer 172.16.1.2 tunnel 1 protocol 'gre'
set vpn ipsec site-to-site peer 172.16.1.2 tunnel 1 remote prefix '10.10.0.2/32'
set vpn ipsec interface 'eth1'

GRE

set interfaces dummy dum0 address '10.10.0.1/32'

set interfaces tunnel tun0 address '10.0.0.1/30'
set interfaces tunnel tun0 encapsulation 'gre'
set interfaces tunnel tun0 source-address '10.10.0.1'
set interfaces tunnel tun0 multicast 'disable'
set interfaces tunnel tun0 remote '10.10.0.2'

VXLAN

set interfaces bridge br0
set interfaces bridge br0 member interface eth2
set interfaces bridge br0 member interface vxlan0
set interfaces vxlan vxlan0 group '239.0.0.241'
set interfaces vxlan vxlan0 source-interface tun0
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 vni '32000'
set interfaces vxlan vxlan0 mtu 1400

Router-2

IPSec

set vpn ipsec site-to-site peer 172.16.1.1 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 172.16.1.1 authentication pre-shared-secret '3mqBSixtMiDy4ngWUKt0uScGH9f2vPoy'
set vpn ipsec site-to-site peer 172.16.1.1 connection-type 'initiate'
set vpn ipsec site-to-site peer 172.16.1.1 ike-group 'aes256-sha256'
set vpn ipsec site-to-site peer 172.16.1.1 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 172.16.1.1 local-address '172.16.1.2'
set vpn ipsec site-to-site peer 172.16.1.1 tunnel 1 esp-group 'aes256-sha256'
set vpn ipsec site-to-site peer 172.16.1.1 tunnel 1 local prefix '10.10.0.2/32'
set vpn ipsec site-to-site peer 172.16.1.1 tunnel 1 protocol 'gre'
set vpn ipsec site-to-site peer 172.16.1.1 tunnel 1 remote prefix '10.10.0.1/32'

set vpn ipsec interface 'eth1'

GRE

set interfaces dummy dum0 address  '10.10.0.2/32'

set interfaces tunnel tun0 address '10.0.0.2/30'
set interfaces tunnel tun0 encapsulation 'gre'
set interfaces tunnel tun0 source-address '10.10.0.2'
set interfaces tunnel tun0 multicast 'disable'
set interfaces tunnel tun0 remote '10.10.0.1'

####VXLAN

set interfaces bridge br0
set interfaces bridge br0 member interface eth2
set interfaces bridge br0 member interface vxlan0
set interfaces vxlan vxlan0 group '239.0.0.241'
set interfaces vxlan vxlan0 source-interface tun0
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 vni '32000'
set interfaces vxlan vxlan0 mtu 1400

Run show command for router-1

vyos@vyos-1# run show vpn ipsec sa
Connection                State    Uptime    Bytes In/Out    Packets In/Out    Remote address    Remote ID    Proposal
------------------------  -------  --------  --------------  ----------------  ----------------  -----------  -------------------------------------
peer_172-16-1-2_tunnel_1  up       2m48s     0B/0B           0/0               172.16.1.2        N/A          AES_CBC_256/HMAC_SHA2_256_128/ECP_256
[edit]

vyos@vyos-1# run show vpn ipsec state

src 172.16.1.1 dst 172.16.1.2
        proto esp spi 0xc6c14624 reqid 1 mode tunnel
        replay-window 0 flag af-unspec
        auth-trunc hmac(sha256) 0xf678038ca4dd81639331b0880be81d8c6c25ec1abbf24ed106df0f098b35aada 128
        enc cbc(aes) 0x30eafcf0d3a58f85fac3c60ed64b4429e0ca2119db03457c85fc84932a94cda6
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 172.16.1.2 dst 172.16.1.1
        proto esp spi 0xc767b48d reqid 1 mode tunnel
        replay-window 32 flag af-unspec
        auth-trunc hmac(sha256) 0x8ccc2770442c281b57598f067edd7101c95bcb8d6448ce13d2949fe7a5941b4e 128
        enc cbc(aes) 0xc141fa9ab4d95e3f23ca03709d633f56908e1b7d693b958a31685b2a8da89587
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 172.16.1.2 dst 172.16.1.1
        proto esp spi 0xcdf788e9 reqid 1 mode tunnel
        replay-window 32 flag af-unspec
        auth-trunc hmac(sha256) 0xb1f5d7bb2095a8e6e9111ec993fab73ba97cbff382958d2fe0e0d510a20c7369 128
        enc cbc(aes) 0xf7b1cc2a8fcc5d90e5481684c6c77b898701ae2aa2cb4132950f772086b4205f
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 172.16.1.2 dst 172.16.1.1
        proto esp spi 0xc4fa5bee reqid 1 mode tunnel
        replay-window 32 flag af-unspec
        auth-trunc hmac(sha256) 0x4cccfcc06f1c5a3942404715850a36d9bf1b6a45a81404590497879de0b44c3b 128
        enc cbc(aes) 0x3cfa48a6b886aa216a856665a58bc853916d23d2158a6c3ca998ea17a0248aea
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 172.16.1.2 dst 172.16.1.1
        proto esp spi 0xc1daf4aa reqid 1 mode tunnel
        replay-window 32 flag af-unspec
        auth-trunc hmac(sha256) 0x271c18b4ee8b1bb69ff146323b625cd44bcc6fcd9bbc8973bfa56fa5bb36ea48 128
        enc cbc(aes) 0xbf76b7d2d1d71fb1756b70abff7de17137255e663a1be45b6488e40b197a0205
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 172.16.1.2 dst 172.16.1.1
        proto esp spi 0xcc1919cd reqid 1 mode tunnel
        replay-window 32 flag af-unspec
        auth-trunc hmac(sha256) 0x6941f9a88157d4ac961ee416d837abf315509ceaa74d70a00ab472ffe4bc439b 128
        enc cbc(aes) 0xb74be7ad9828769a16c2a1a20bfdd6dcfec951bd25dabfe181127233322746c7
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 172.16.1.2 dst 172.16.1.1
        proto esp spi 0xc76ca308 reqid 1 mode tunnel
        replay-window 32 flag af-unspec
        auth-trunc hmac(sha256) 0x0a808f172fbf9b4e019d266dc27ccc6d22f9e1a7ea68a0a9b72d620b4a5de988 128
        enc cbc(aes) 0x8b32c718060e991a773e21c596499fd5cf01025e436d59536cac17a58578f660
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
[edit]

vyos@vyos-1# run show vpn ipsec status
IPSec Process Running: 56958
Security Associations (1 up, 0 connecting):
peer_172-16-1-2[2]: ESTABLISHED 4 minutes ago, 172.16.1.1[172.16.1.1]...172.16.1.2[172.16.1.2]
peer_172-16-1-2_tunnel_1{261}:  DELETED, TUNNEL, reqid 1
peer_172-16-1-2_tunnel_1{261}:   10.10.0.1/32[gre] === 10.10.0.2/32[gre]
peer_172-16-1-2_tunnel_1{262}:  DELETED, TUNNEL, reqid 1
peer_172-16-1-2_tunnel_1{262}:   10.10.0.1/32[gre] === 10.10.0.2/32[gre]
peer_172-16-1-2_tunnel_1{263}:  DELETED, TUNNEL, reqid 1
peer_172-16-1-2_tunnel_1{263}:   10.10.0.1/32[gre] === 10.10.0.2/32[gre]
peer_172-16-1-2_tunnel_1{264}:  DELETED, TUNNEL, reqid 1
peer_172-16-1-2_tunnel_1{264}:   10.10.0.1/32[gre] === 10.10.0.2/32[gre]
peer_172-16-1-2_tunnel_1{265}:  DELETED, TUNNEL, reqid 1
peer_172-16-1-2_tunnel_1{265}:   10.10.0.1/32[gre] === 10.10.0.2/32[gre]
peer_172-16-1-2_tunnel_1{266}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: cf6fde0e_i cc31c995_o
peer_172-16-1-2_tunnel_1{266}:   10.10.0.1/32[gre] === 10.10.0.2/32[gre]
[edit]

Note:
Router-2 has the same output.

Problem:
I could not ping tun0 interface of the router-2.

vyos@vyos-1# ping 10.0.0.2 -c 5
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.

--- 10.0.0.2 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4097ms

Also, I could not ping Host-1 from another host-2. Could you help me resolve this problem, please?

Thank you very much in advance. I would appreciate any help.

Viacheslav · October 12, 2021, 2:17pm

ESP lifetime not should be more than IKE lifetime.

omidit · October 12, 2021, 2:28pm

Thank you for your reply. I changed it from 28800 to 3500. But it still does not work.

n.fort · October 12, 2021, 3:32pm

Hi @omidit .
In your config, you should change tunnel source-address and remote address, and use “public” IP address (172.16.1.X in your case).
Refer to: Tunnel — VyOS 1.4.x (sagitta) documentation

omidit · October 13, 2021, 11:14am

Hi,
Thank you so much, I changed the tunnel source address and remote address, so it works, thanks.

Router-1

GRE

set interfaces tunnel tun0 address '10.0.0.1/30'
set interfaces tunnel tun0 encapsulation 'gre'
set interfaces tunnel tun0 source-address '172.16.1.1'
set interfaces tunnel tun0 multicast 'disable'
set interfaces tunnel tun0 remote '172.16.1.2'

Router-2

GRE

set interfaces tunnel tun0 address '10.0.0.2/30'
set interfaces tunnel tun0 encapsulation 'gre'
set interfaces tunnel tun0 source-address '172.16.1.2'
set interfaces tunnel tun0 multicast 'disable'
set interfaces tunnel tun0 remote '172.16.1.1'

Now, I have a problem with VXLAN. Although the VXLAN interface (vxlan0) in both routers can receive the ping traffic over the tunnel, hosts can not ping each other.
if I ping host 2 from host 1. The output of the run show command is:

Router-1

vyos@vyos-1# run show interfaces vxlan vxlan0
vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
    link/ether 22:40:92:ca:46:36 brd ff:ff:ff:ff:ff:ff

    RX:  bytes  packets  errors  dropped  overrun       mcast
          0      0       0        0        0           0
    TX:  bytes  packets  errors  dropped  carrier  collisions
          6980      150       0        0        0           0

Router-2

vyos@vyos-2# run show interfaces vxlan vxlan0
vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
    link/ether 52:a6:7d:53:a5:48 brd ff:ff:ff:ff:ff:ff

    RX:  bytes  packets  errors  dropped  overrun       mcast
          6704      144       0        0        0           0
    TX:  bytes  packets  errors  dropped  carrier  collisions
          0       0       0        0        0           0

Also, I can see ARP requests and replies in both hosts, Thanks to help me to solve the problem.

16again · October 13, 2021, 8:26pm

Why can’t you use loopback address as GRE source?
I’ve been succesful using LAN interface IP as GRE tunnel source.

When you use WAN IPs, and ipsec tunnel fails for some reason, you’re sending unencrypted GRE packets. Moreover, some are on DHCP , using DDNS + IPSEC, which rules out using WAN ip as GRE source.

fernando · October 14, 2021, 5:34pm

Hi

I think that there is a problem with your vxlan’s configuration , because it shows that you need set multicast to establish the tunnel and currently gre settings they are disabling.

set interfaces tunnel tun0 multicast 'disable'

vxlan configuration:

set interfaces vxlan vxlan0 group '239.0.0.241'
set interfaces vxlan vxlan0 source-interface tun0
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 vni '32000'
set interfaces vxlan vxlan0 mtu 140

also if you want to check in our documentation ,it’s possible to set the tunnel with unicast instead of multicast, I share the link below:
https://docs.vyos.io/en/equuleus/configuration/interfaces/vxlan.html

system · October 16, 2021, 5:35pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.