Route-Based Site-to-Site VPN to Azure (BGP over IPsec)

Ali · October 13, 2020, 12:07am

Hello Everyone,
I’ve been working on setting up a VyOS (1.3.x) site-to-site VPN tunnel to my Azure resource cloud, I used this link and when I was configuring I lost my internet access to all of sites from VyOS :

admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication id ‘y.y.y.y’
set vpn ipsec site-to-site peer x.x.x.x authentication remote-id ‘x.x.x.x’
set vpn ipsec site-to-site peer x.x.x.x connection-type ‘respond’
set vpn ipsec site-to-site peer x.x.x.x description ‘AZURE TUNNEL’
set vpn ipsec site-to-site peer x.x.x.x ike-group ‘AZUREI’
set vpn ipsec site-to-site peer x.x.x.x ikev2-reauth ‘inherit’
set vpn ipsec site-to-site peer x.x.x.x local-address ‘10.130.90.1’
set vpn ipsec site-to-site peer x.x.x.x vti bind ‘vti1’
set vpn ipsec site-to-site peer x.x.x.x vti esp-group ‘AZURE’[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication mode ‘pre-shared-secret’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication pre-shared-secret ‘**********’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication remote-id ‘x.x.x.x’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x connection-type ‘respond’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x description ‘AZURE TUNNEL’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x ike-group ‘AZURE’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x ikev2-reauth ‘inherit’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x local-address ‘10.130.90.1’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x vti bind ‘vti1’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x vti esp-group ‘AZURE’
[edit]
admin@vyos# commit
[ vpn ]
Warning: Local address 10.130.90.1 specified for peer “x.x.x.x”
is not configured on any of the ipsec-interfaces and is not the
clustering address. IPsec must be re-started after address
has been configured.

My Lan (eth0) interfaces are:
10.130.90.1/24
10.130.0.1/24
10.130.90.254/24

x.x.x.x is Azure VNet Gateway public IP
y.y.y.y is my public IP address

I also set a public IP on WAN interface (eth1), I think I lost my internet after I used:
set vpn ipsec site-to-site peer “x.x.x.x” local-address ‘10.130.90.1’
I used 10.130.90.1 as a private IP address.
do you have any ideas guys ?

Regards

Dmitry · October 13, 2020, 12:05pm

Hello @Ali, can you try to add

set vpn ipsec options disable-route-autoinstall

Did you check IPSec connectivity?

Ali · October 13, 2020, 11:51pm

Hi @Dmitry no I didn’t check because after I got this messagfe I lost connection to network and sites so I had to restored last vyos snapshot again to fix it.

admin@vyos# commit
[ vpn ]
Warning: Local address 10.130.90.1 specified for peer “x.x.x.x”
is not configured on any of the ipsec-interfaces and is not the
clustering address. IPsec must be re-started after address
has been configured.

Do I need to add it first ?

Dmitry · October 14, 2020, 7:48am

I think yes. But why you can’t get access to the remote console?

Ali · October 14, 2020, 8:10am

We have a Vcenter (located in datacenter) that everything connected to the VyOS firewall, we have just one firewall and I have remote access to the VM on Vcenter so we don’t have access if firewall goes down! maybe I need to have second firewall ?!
last time I lost everything and asked Datacenter to restore last snapshot … because of this and I need to make sure about configuration before do anything.

Ali · October 19, 2020, 12:18am

Hi @Dmitry, thanks for your help again
I wanna try to setup it again but I have some question before do it, I wanted to know if I can’t resolve the problem with that command, how can I check all parameters to find the problem on VyOS ?
do you have any idea ?

Viacheslav · October 19, 2020, 6:34am

Are you sure that you have that ip address on some ethX interface?
Also, you need to add option “ipsec interface”

set vpn ipsec ipsec-interfaces interface ethX

Ali · October 19, 2020, 7:37am

Yes I have this IP address as one of my eth0 IP’s !
I also have another IPsec tunnel and I used
set vpn ipsec ipsec-interfaces interface ethX

Dmitry · October 19, 2020, 8:41am

Hi @Ali, do you have any chance to save remote access to the remote router after enabling IPSec?

Ali · October 20, 2020, 2:22am

Hi @Dmitry and @Viacheslav
I want to put all of my configuration again, I have a Vcenter with some VM machines and one VyOS firewall on it, I have one IPsec tunnel to another site and because of this I activate IPsec on it (eth1) before and it’s working fine so I added this before:
set vpn ipsec ipsec-interfaces interface 'eth1'
Now I’m trying to have a VPN tunnel between Azure and VyOS again, I used this link

my configuration on VyOS:
# IKE Config
set vpn ipsec esp-group AZURE compression ‘disable’
set vpn ipsec esp-group AZURE lifetime ‘3600’
set vpn ipsec esp-group AZURE mode ‘tunnel’
set vpn ipsec esp-group AZURE pfs ‘dh-group2’
set vpn ipsec esp-group AZURE proposal 1 encryption ‘aes256’
set vpn ipsec esp-group AZURE proposal 1 hash ‘sha1’
set vpn ipsec ike-group AZURE dead-peer-detection action ‘restart’
set vpn ipsec ike-group AZURE dead-peer-detection interval ‘15’
set vpn ipsec ike-group AZURE dead-peer-detection timeout ‘30’
set vpn ipsec ike-group AZURE ikev2-reauth ‘yes’
set vpn ipsec ike-group AZURE key-exchange ‘ikev2’
set vpn ipsec ike-group AZURE lifetime ‘28800’
set vpn ipsec ike-group AZURE proposal 1 dh-group ‘2’
set vpn ipsec ike-group AZURE proposal 1 encryption ‘aes256’
set vpn ipsec ike-group AZURE proposal 1 hash ‘sha1’

# Dummy VTI IP
set interfaces vti vti1 address '10.130.100.1/32'
set interfaces vti vti1 description 'Azure Tunnel'

# MSS
set firewall options interface vti1 adjust-mss 1350

# IPSEC Config
set vpn ipsec site-to-site peer 62.255.61.23 authentication id 'x.x.x.x'
set vpn ipsec site-to-site peer 62.255.61.23 authentication mode 'pre-shared-secret' 
set vpn ipsec site-to-site peer 62.255.61.23 authentication pre-shared-secret '***********'
set vpn ipsec site-to-site peer 62.255.61.23 authentication remote-id '62.255.61.23'
set vpn ipsec site-to-site peer 62.255.61.23 connection-type 'respond'
set vpn ipsec site-to-site peer 62.255.61.23 description 'AZURE TUNNEL'
set vpn ipsec site-to-site peer 62.255.61.23 ike-group 'AZURE'
set vpn ipsec site-to-site peer 62.255.61.23 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 62.255.61.23 local-address '10.130.90.1'
set vpn ipsec site-to-site peer 62.255.61.23 vti bind 'vti1'
set vpn ipsec site-to-site peer 62.255.61.23 vti esp-group 'AZURE'

# BGP Listener
set protocols static interface-route 10.10.0.254/32 next-hop-interface vti1

# BGP Settings
set protocols bgp 65534 neighbor 10.10.0.254 remote-as '65517'
set protocols bgp 65534 neighbor 10.10.0.254 address-family ipv4-unicast soft-reconfiguration 'inbound'
set protocols bgp 65534 neighbor 10.10.0.254 timers holdtime '30'
set protocols bgp 65534 neighbor 10.10.0.254 timers keepalive '10'

# Connection Check Disable
set protocols bgp 65532 neighbor 10.10.0.254 disable-connected-check

My Lan (eth0) interfaces are:
10.130.90.1/24
10.130.0.1/24
10.130.90.254/24

I also have x.x.x.x as a public IP address on WAN (eth1) Interface.

Viacheslav · October 20, 2020, 4:23am

Check your Azure policy so that ports 500, 4500 are open.
Can you provide some tunnels/vti states/logs?

Ali · October 20, 2020, 5:33am

Yes, in azure everything is fine.
when I was configuring I lost my network access and had to restore last snapshot to have everything again so I didn’t have any chance to see logs!
my question is why I lost everything after add commands and how can I fix it.

it’s happened after this step:

admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication id ‘y.y.y.y’
set vpn ipsec site-to-site peer x.x.x.x authentication remote-id ‘x.x.x.x’
set vpn ipsec site-to-site peer x.x.x.x connection-type ‘respond’
set vpn ipsec site-to-site peer x.x.x.x description ‘AZURE TUNNEL’
set vpn ipsec site-to-site peer x.x.x.x ike-group ‘AZUREI’
set vpn ipsec site-to-site peer x.x.x.x ikev2-reauth ‘inherit’
set vpn ipsec site-to-site peer x.x.x.x local-address ‘10.130.90.1’
set vpn ipsec site-to-site peer x.x.x.x vti bind ‘vti1’
set vpn ipsec site-to-site peer x.x.x.x vti esp-group ‘AZURE’[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication mode ‘pre-shared-secret’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication pre-shared-secret ‘**********’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x authentication remote-id ‘x.x.x.x’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x connection-type ‘respond’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x description ‘AZURE TUNNEL’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x ike-group ‘AZURE’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x ikev2-reauth ‘inherit’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x local-address ‘10.130.90.1’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x vti bind ‘vti1’
[edit]
admin@vyos# set vpn ipsec site-to-site peer x.x.x.x vti esp-group ‘AZURE’
[edit]
admin@vyos# commit
[ vpn ]
Warning: Local address 10.130.90.1 specified for peer “x.x.x.x”
is not configured on any of the ipsec-interfaces and is not the
clustering address. IPsec must be re-started after address
has been configured.

I also have 10.130.90.1/24 on my eth0 (LAN) and public ip (x.x.x.x) is on my eth1 (WAN).
in my VyOS Fw I have another Ipsec for another site that working fine and set on my eth1:

set vpn ipsec ipsec-interfaces interface ‘eth1’

Viacheslav · October 20, 2020, 7:04am

admin@vyos# set vpn ipsec site-to-site peer x.x.x.x local-address ‘x.x.x.x’

Here you need to declare the local address (WAN interface) that will initiate/respond/communicate with the Azure (WAN) site.

Ali · October 20, 2020, 11:15pm

Hi @Viacheslav
I copied this from sample configuration:

set vpn ipsec site-to-site peer 203.0.113.2 local-address ‘10.10.0.5’

do I need to change it to:

set vpn ipsec site-to-site peer x.x.x.x local-address ‘x.x.x.x’

x.x.x.x means my public ip address that configure on WAN interface (eth1)

I added:

set vpn ipsec site-to-site peer 62.255.61.23 local-address ‘10.130.90.1’

62:255.61.23 is Azure VNet Gateway public IP
10.130.90.1 is my Vyos private IP that configured on Eth0 (LAN)

Ali · October 21, 2020, 9:20am

Hi @Dmitry and @Viacheslav
first thanks for your help, I wanna setup it again now so do you have any recommendations ?
I wanna just try this:

set vpn ipsec options disable-route-autoinstall

is there anything else that can help me to solve this problem ?
Regards

Ali · October 22, 2020, 12:52am

Hi @Viacheslav
Finally I fixed it, problem was :

admin@vyos# set vpn ipsec site-to-site peer y.y.y.y local-address ‘x.x.x.x’

x.x.x.x = should be public IP address of VyOS
y.y.y.y = it’s Azure VNet Gateway public IP

thanks so much for your support again

system · October 24, 2020, 12:52am

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.