We’re in the planning phase to try and move from Watchguard to VyOS or pfSense. So I’m building a list of features we use all the time, and basic things we setup and don’t even think about, to test with VyOS before considering it an option. While I think most of the features are a no brainer, I think I found an option that isn’t workable in VyOS.
In our IPSEC tunnels for Watchguard, both Policy-based or Virtual Interface/Routing-based, I can specify multiple gateways (so WAN1 to remote, then WAN2 to remote) for failover service. But in VyOS you specify which interface the Peer Tunnel is to use, is there a way to allow WAN failover?
And if I think about it more, also multiple remotes? For a couple clients, they might have multiple sites, and each site has 2 ISPs. So the IKE/Phase 1 gateway list might look like:
- Local WAN1 to Remote WAN1
- Local WAN2 to Remote WAN1
- Local WAN1 to Remote WAN2
- Local WAN2 to Remote WAN2
Is that possible as well?
It possible, but you need some Local PBR rules. In these rules you need to declare from which interface /source ip you want to communicate to each peer site. Also you need to understand what you want if one interface will be down.
Hi, Viacheslav
I have the same like request. I setup vyos with 2 or more wireguard tunnel, I can set PBR to let lan(like 192.168.0.0/24) network default gateway surfing internet with one wireguard tunnel. But. sometimes it will down. So I want to let wan load balancing work with PBR to let 2 wireguard work in load balancing mode. How should i do ?
Can you provide a simple network diagram?
For example, R1 via 2 ISP’s has a connection with R4 and R5 or R4 have also 2 isp and you need to communicate with remote one site via 2 ISP (remote site also has 2 isp)? I attached an example of a map.
And what do you want to use IPsec or wireguard?
So in my scenario I have the following rough layout:
I’m doing IPSEC Site-to-Site VPN only.
My Local VyOS site has 2 ISPs. ISPA is on eth0, ISPB is on eth1. Static IPs on both.
My Remote Watchguard site has 2 ISPs. ISPA is on eth0, ISPC is on eth1. Static IPs on both.
The Remote Unknown site is a hosted AWS instance from a third-party, they are sending me config information for a single endpoint tunnel.
From my Local VyOS a Site-to-Site IPSEC VPN via ISPA (1 WAN at a time) is easy, but I’m wanting to make the IPSEC tunnel automatically failover to ISPB.
The Watchguard Branch Office VPN does this fairly easy, in their Gateway settings, you just list additional local gateways and remote gateways.
If the failover can be figured out, I assume my second question (Local ISPA to Remote ISPA, failover Local ISPB to Remote ISPA, failover Local ISPB to Remote ISPA, failover Local ISPB to Remote ISPB) will be possible.
An example of how this can be done with VyOS
We need 4 ipip tunnels on each site
- R1 isp01 - R2 isp01
- R1 isp01 - R2 isp02
- R1 isp02 - R2 isp01
- R1 isp02 - R2 isp02
And over the ipip tunnel, we build IPSEC with vti interfaces.
Additional PBR local rules for correct respond
R1
sudo ip rule add from 203.0.113.1 lookup 100
sudo ip rule add from 198.51.100.1 lookup 101
R2
sudo ip rule add from 203.0.113.254 lookup 100
sudo ip rule add from 198.51.100.254 lookup 101
I attached configuration from R1 and R2
R1-ipsec-multivan.txt (5.9 KB)
R2-ipsec-multivan.txt (5.9 KB)
vyos@r1:~$ show vpn ipsec sa
Connection State Up Bytes In/Out Remote address Remote ID Proposal
--------------------------- ------- --------- -------------- ---------------- ----------- ----------------------------------------
peer-192.168.1.2-tunnel-vti up 7 minutes 0B/0B 192.168.1.2 N/A AES_GCM_16_256/PRF_HMAC_SHA2_256/ECP_256
peer-192.168.2.2-tunnel-vti up 7 minutes 0B/0B 192.168.2.2 N/A AES_GCM_16_256/PRF_HMAC_SHA2_256/ECP_256
peer-192.168.3.2-tunnel-vti up 7 minutes 0B/0B 192.168.3.2 N/A AES_GCM_16_256/PRF_HMAC_SHA2_256/ECP_256
peer-192.168.0.2-tunnel-vti up 7 minutes 0B/0B 192.168.0.2 N/A AES_GCM_16_256/PRF_HMAC_SHA2_256/ECP_256
While your configuration appears to complete Multi-WAN VPN, what about VyOS IPSEC to non-VyOS systems? I control one remote side (Watchguard), but not the other remote side (AWS). Or am I mis-understanding something?
Can you specify if you mean one or two tunnels with a remote side?
Sorry the last image was so sparse, here’s more of what I’m trying to do. Right now we use Watchguard everywhere, and it’s been great. But I have the opportunity to replace a few units coming up, so I’m running a couple labs to verify I have the same features needed. Mostly basic NAT, routing, port forwarding, but the VPN failing over with multiple gateways is highly used for almost all my clients.
Recently we setup a new AWS tunnel with a new third-party group and the config they sent me was 2 different tunnels setup using metric 1 for the first tunnel and metric 2 for the second tunnel, using the same routing/endpoint. That setup might be an option if that’s how we have to do things. But I’m used to how Watchguard does it, with multiple gateways for a single tunnel.
I just got the Local VyOS online with public routing information, so I will be doing more in depth testing soon, I can try to give you attempted configurations soon.
So I was working on this today and I have a single WAN on my Local VyOS connected to a single WAN on a remote Watchguard. Slightly different scenario, but would come up; the remote Watchguard I was working with is using an SDWAN appliance to load balance the ISPs, so it only has a single IP and single WAN interface on the firewall.
Here’s the working VyOS config:
# Firewall
set firewall all-ping 'enable'
set firewall broadcast-ping 'disable'
set firewall config-trap 'disable'
set firewall group address-group MAIN_OFF address '8.8.8.2'
set firewall group address-group MAIN_OFF address '7.7.7.7'
set firewall group port-group MAIN_OFF-SUPPORT port '22'
set firewall ipv6-receive-redirects 'disable'
set firewall ipv6-src-route 'disable'
set firewall ip-src-route 'disable'
set firewall log-martians 'enable'
set firewall name WAN-LAN default-action 'drop'
set firewall name WAN-LAN rule 10 action 'accept'
set firewall name WAN-LAN rule 10 state established 'enable'
set firewall name WAN-LAN rule 10 state related 'enable'
set firewall name WAN-LOCAL default-action 'drop'
set firewall name WAN-LOCAL rule 10 action 'accept'
set firewall name WAN-LOCAL rule 10 state established 'enable'
set firewall name WAN-LOCAL rule 10 state related 'enable'
set firewall name WAN-LOCAL rule 20 action 'accept'
set firewall name WAN-LOCAL rule 20 icmp type-name 'echo-request'
set firewall name WAN-LOCAL rule 20 protocol 'icmp'
set firewall name WAN-LOCAL rule 20 state new 'enable'
set firewall name WAN-LOCAL rule 30 action 'accept'
set firewall name WAN-LOCAL rule 30 destination group port-group 'MAIN_OFF-SUPPORT'
set firewall name WAN-LOCAL rule 30 log 'enable'
set firewall name WAN-LOCAL rule 30 source group address-group 'MAIN_OFF'
set firewall name WAN-LOCAL rule 30 state new 'enable'
set firewall name WAN-LOCAL rule 40 action 'accept'
set firewall name WAN-LOCAL rule 40 description 'Accept ESP'
set firewall name WAN-LOCAL rule 40 log 'enable'
set firewall name WAN-LOCAL rule 40 protocol 'esp'
set firewall name WAN-LOCAL rule 40 state new 'enable'
set firewall name WAN-LOCAL rule 41 action 'accept'
set firewall name WAN-LOCAL rule 41 description 'Accept ISAKMP'
set firewall name WAN-LOCAL rule 41 destination port '500'
set firewall name WAN-LOCAL rule 41 log 'enable'
set firewall name WAN-LOCAL rule 41 protocol 'udp'
set firewall name WAN-LOCAL rule 41 state new 'enable'
set firewall receive-redirects 'disable'
set firewall send-redirects 'enable'
set firewall source-validation 'disable'
set firewall syn-cookies 'enable'
set firewall twa-hazards-protection 'disable'
# Interfaces
set interfaces ethernet eth0 address '10.10.10.1/24'
set interfaces ethernet eth0 description 'LAN1'
set interfaces ethernet eth1 description 'LAN2'
set interfaces ethernet eth2 address '2.2.2.2/28'
set interfaces ethernet eth2 description 'WAN1_CTL'
set interfaces ethernet eth2 firewall in name 'WAN-LAN'
set interfaces ethernet eth2 firewall local name 'WAN-LOCAL'
set interfaces ethernet eth3 address '1.1.1.2/28'
set interfaces ethernet eth3 description 'WAN2_MDC'
set interfaces ethernet eth3 firewall in name 'WAN-LAN'
set interfaces ethernet eth3 firewall local name 'WAN-LOCAL'
set interfaces vti vti0 address '169.254.56.0/31'
set interfaces vti vti0 description 'Test VTI'
# NAT
set nat source rule 10 destination address '192.168.10.0/24'
set nat source rule 10 exclude
set nat source rule 10 outbound-interface 'eth2'
set nat source rule 10 source address '10.10.10.0/24'
set nat source rule 11 destination address '192.168.10.0/24'
set nat source rule 11 exclude
set nat source rule 11 outbound-interface 'eth3'
set nat source rule 11 source address '10.10.10.0/24'
set nat source rule 100 outbound-interface 'eth2'
set nat source rule 100 source address '10.10.10.0/24'
set nat source rule 100 translation address 'masquerade'
# Static Routing
set protocols static route 0.0.0.0/0 next-hop 1.1.1.1
set protocols static route 0.0.0.0/0 next-hop 2.2.2.1
set protocols static route 192.168.10.0/24 next-hop 169.254.56.1 distance '1'
# VPN
set vpn ipsec esp-group VTI0_Phase2 lifetime '3600'
set vpn ipsec esp-group VTI0_Phase2 mode 'tunnel'
set vpn ipsec esp-group VTI0_Phase2 pfs 'disable'
set vpn ipsec esp-group VTI0_Phase2 proposal 1 encryption 'aes256'
set vpn ipsec esp-group VTI0_Phase2 proposal 1 hash 'sha1'
set vpn ipsec ike-group VTI0_Phase1 key-exchange 'ikev1'
set vpn ipsec ike-group VTI0_Phase1 lifetime '86400'
set vpn ipsec ike-group VTI0_Phase1 proposal 1 dh-group '2'
set vpn ipsec ike-group VTI0_Phase1 proposal 1 encryption 'aes256'
set vpn ipsec ike-group VTI0_Phase1 proposal 1 hash 'sha1'
set vpn ipsec ipsec-interfaces interface 'eth3'
set vpn ipsec site-to-site peer 5.5.5.5 authentication id '1.1.1.14'
set vpn ipsec site-to-site peer 5.5.5.5 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 5.5.5.5 authentication pre-shared-secret 'vyos2WATCHGUARD'
set vpn ipsec site-to-site peer 5.5.5.5 authentication remote-id '5.5.5.5'
set vpn ipsec site-to-site peer 5.5.5.5 connection-type 'respond'
set vpn ipsec site-to-site peer 5.5.5.5 ike-group 'VTI0_Phase1'
set vpn ipsec site-to-site peer 5.5.5.5 local-address '1.1.1.14'
set vpn ipsec site-to-site peer 5.5.5.5 vti bind 'vti0'
set vpn ipsec site-to-site peer 5.5.5.5 vti esp-group 'VTI0_Phase2'
Here’s the working Watchguard Config:
When doing Watchguard to Watchguard, I would just add a second, third, or fourth gateway to the list and the tunnel(s) would fail-over depending on which gateway pair was online. If I am working with a remote Watchguard with 2 or more ISPs, as I originally stated, then I should be able to add multiple ipsec-interfaces and multiple site-to-site peer tunnels using those remote IPs. But for a site using a single ISP, or using some sort of load balancer/SDWAN, I cannot specify the same peer IP for a second tunnel/gateway. Correct?
If you need 2 tunnel for same ip, it works only with policy based VPN
You can map ip to FQDN
set system static-host-mapping host-name foo1.local inet 5.5.5.5
set system static-host-mapping host-name foo2.local inet 5.5.5.5
set vpn ipsec site-to-site peer foo1.local xxxxx
set vpn ipsec site-to-site peer foo2.local xxxxx
Some new features will be with XFRM interfaces ref Route-based VPN :: strongSwan Documentation
