I want to setup 2 wireguard tunnels on vyos in lan and 2 wireguard connected to remote wireguard vpn server on cloud.
Currently I set PBR to let LAN(192.168.2.0/22) default gateway to wg01 interfaces. so all computers in LAN can surfing internet by wg01 tunnel, or i can setup wg02 as the default gateway for LAN.
The question is how to make wan load balancing between wg01 and wg02 , then if one tunnel is broken ,our internet access is fine.
Do you have 1 or 2 ISP on vyos LAN node?
Currently only 1 ISP.
The main problem is that Wireguard interfaces are in UP state even though no real connection to the remote site is established. For example if you disble link on cloud1, vyos node wg0 interface will be in up state…
How about use bgp for load balancing and failover?
Cloud1 and Cloud2 use bgp for connection with node “vyos” via wireguard interfaces. Each cloud export “default originate” route to “vyos”
VyOS places these routes to table 100.
One problem, that we don’t have a native cli for placing prefixes to route-table 100. So we need some tricks yet.
Configuration “vyos”:
set interfaces ethernet eth0 address '100.64.0.2/30'
set interfaces ethernet eth1 address '192.168.2.1/24'
set interfaces ethernet eth1 policy route 'PBR'
set interfaces wireguard wg0 address '10.10.10.254/24'
set interfaces wireguard wg0 description 'Server-CLOUD1-203.0.113.254'
set interfaces wireguard wg0 peer CLOUD1 allowed-ips '10.10.10.0/24'
set interfaces wireguard wg0 peer CLOUD1 allowed-ips '0.0.0.0/0'
set interfaces wireguard wg0 peer CLOUD1 endpoint '203.0.113.254:64216'
set interfaces wireguard wg0 peer CLOUD1 pubkey 'xx='
set interfaces wireguard wg1 address '10.20.20.254/24'
set interfaces wireguard wg1 description 'Server-CLOUD2-198.51.100.254'
set interfaces wireguard wg1 peer CLOUD2 allowed-ips '10.20.20.0/24'
set interfaces wireguard wg1 peer CLOUD2 allowed-ips '0.0.0.0/0'
set interfaces wireguard wg1 peer CLOUD2 endpoint '198.51.100.254:64216'
set interfaces wireguard wg1 peer CLOUD2 pubkey 'xx='
set nat source rule 100 outbound-interface 'wg0'
set nat source rule 100 source address '192.168.2.0/24'
set nat source rule 100 translation address 'masquerade'
set nat source rule 110 outbound-interface 'wg1'
set nat source rule 110 source address '192.168.2.0/24'
set nat source rule 110 translation address 'masquerade'
set policy route PBR rule 10 destination address '!192.168.2.0/24'
set policy route PBR rule 10 set table '100'
set policy route PBR rule 10 source address '192.168.2.0/24'
set protocols bgp 65001 maximum-paths ibgp '2'
set protocols bgp 65001 neighbor 10.10.10.1 remote-as '65001'
set protocols bgp 65001 neighbor 10.10.10.1 timers holdtime '30'
set protocols bgp 65001 neighbor 10.10.10.1 timers keepalive '10'
set protocols bgp 65001 neighbor 10.10.10.1 update-source '10.10.10.254'
set protocols bgp 65001 neighbor 10.20.20.1 remote-as '65001'
set protocols bgp 65001 neighbor 10.20.20.1 timers holdtime '30'
set protocols bgp 65001 neighbor 10.20.20.1 timers keepalive '10'
set protocols bgp 65001 neighbor 10.20.20.1 update-source '10.20.20.254'
set protocols static interface-route 10.10.10.0/24 next-hop-interface wg0
set protocols static interface-route 10.20.20.0/24 next-hop-interface wg1
set protocols static route 198.51.100.254/32 next-hop 100.64.0.1
set protocols static route 203.0.113.254/32 next-hop 100.64.0.1
Additional tricks:
sudo vtysh -c "conf t" -c "route-map RMAP-IN permit 100" -c " set table 100"
sudo vtysh -c "conf t" -c "router bgp 65001" -c "address-family ipv4 unicast" -c "neighbor 10.10.10.1 route-map RMAP-IN in" -c "neighbor 10.20.20.1 route-map RMAP-IN in"
So we receive 2 default routes from neighbors
vyos@vyos:~$ show ip bgp sum
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.10.10.1 4 65001 148 155 0 0 0 00:13:31 1
10.20.20.1 4 65001 71 70 0 0 0 01:00:56 1
And these routes exported to table 100
vyos@vyos:~$ show ip route table 100
VRF default table 100:
B>* 0.0.0.0/0 [200/0] via 10.10.10.1, wg0, 00:00:10
* via 10.20.20.1, wg1, 00:00:10
vyos@vyos:~$
I reduced the timers, for a faster response of the bgp states.
Feature request for “set table x” in route-maps ⚓ T3032 Ability to "set table" in the policy route-map
Update.
That feature will be available in the next rolling release.
set policy route-map RMAP-IN rule 100 action permit
set policy route-map RMAP-IN rule 100 set table 100Great! Thank you so much. I’ll try it.
Does this require control of the Cloud based WG instances to use BGP (sorry for the basic question, this is new to me)?
I ask as I would like to achieve load-balanced routes via WG to some mullvad VPN endpoints, obviously I have no control over those whatsoever.
You can deploy VyOS in almost any cloud.
Google, Amazon, Azure, Packet, etc. or something of your own.
This is just one solution, there are other options to consider.
Scale of service offering wins out here though. For $5 a month I get access to any one of hundreds of servers mullvad have deployed globally, along with no data cap/costs. Individually I just can’t compete with that.
Sorry, I don’t use mullvad. I can’t say what options are available there.
@echowings sorry for hijacking your thread but apparently I have the same question and the answer is still not clear to me.
@Viacheslav it would be glad if you could clarify below topic once again.
I’m routing several VLANs through wireguard tunnel. Is it possible to add second wireguard tunnel and configure load balancing or at least automatic fail-over when 2nd tunnel starts accepting traffic if the first one is experiencing high latency or down? Assuming that I have no control over wireguard servers since they’re managed by 3rd-party VPN provider.
The goal is very simple, obviously if you route all traffic from VLAN through VPN, you’d like to add some redundancy.
If this is tricky with wireguard, is it possible with IPsec or OpenVPN? Again, assuming that I won’t have control over servers.
I’m currently using Opnsense where this all possible but it has other problems. So I’m considering Vyos and trying to figure out if it fits my needs.
@vala4i What you receive over remote site/sites? Only the default route? Do you use any routing protocol? From your side one IP address and 2 tunnels with the remote site?
Can you describe more details?
I believe currently I just point default route to the tunnel and I’m not using any BGP/OSPF stuff. This is just a home use scenario nothing too advanced.
Assume that I live in Europe but would like to access Netflix content available for US users only (Netflix performs filtering based on IP geo location). So, my plan is to create a separate VLAN for my smart TV and route all the traffic from that VLAN through VPN server located in the US. For simplicity, I’m routing all public traffic from that VLAN via VPN not just Netflix subnets. And again to reduce maintenance overhead I just buy access to the VPN server from a 3rd-party provider and don’t manage servers myself.
Network topology is very basic: ISP <-> Vyos router <-> LAN. VLAN is being terminated on Vyos router.
Obviously when VPN is down I loose access to Netflix, since from my Europe IP US content won’t be available. So, I’d like to add some redundancy here. And the plan is to keep 2-3 VPN tunnels running and either load balance connections between them or at least utilize automatic failover feature when default route in the VLAN will be switched to the secondary tunnel if primary is having issues.
Ideally I’d like to make it work with Wireguard VPN but if this isn’t an option I’d be glad to hear if this is possible with OpenVPN or IPsec.
Thanks a lot for your time @Viacheslav!
Try wan load-balancing Load-balancing — VyOS 1.4.x (sagitta) documentation
Gonna look into that. Thanks!
Hi, Viacheslav
Does the vyos 1.2.7 release version support this command ?
@echowings Sure
vyos@r2-lts# set protocols bgp 65001 maximum-paths
Possible completions:
ebgp Maximum ebgp multipaths
ibgp Maximum ibgp multipathsGreat! Thas.
Another question。 How to setup mutiwan routing with bgp routing ?I have 5 wan ,and I want to let bgp routing to make policy base routing . for example google ASN surfing internet via WAN 1 and WAN2(Loding blance), Microsoft ASN surfing internete via WAN 2 and 3(loading blance),How to set routing policy bas BGP?
Sorry, I mean this feature:
set policy route-map RMAP-IN rule 100 action permit
set policy route-map RMAP-IN rule 100 set table 100
Does it work on vyos 2.7 release version?