What is the best vpn protocol to use for site to site on vyos?

Hello.
Our vyos deployment has been a resounding success so far. :smile:
However, now i’d like to start using some additional services like vpn.
I’m wondering, what’s the best vpn protocol to use on vyos? it looks like it supports most of the popular protocols. I have a set of 4 locations that should be able to talk to one another directly, site to site without having to route through the vpn server. What should i be aware of when setting up such a thing?

This is a simplified overview to start the discussion, i’d like to know what people’s thoughts are and how others more experienced than me would do this.

Thanks.

Well it depends on a few things:

  • dynamic routing. Do you plan to use a routing protocol? With 4 locations, depending on your data flows, its worth considering
  • NAT : Do you have your VPN endpoints behind a NAT or not ? IPSEC doesn’t like NAT, although it still can work when one of the endpoint is behingd NAT
  • bandwidth : OpenVPN works for bandwidth around 10Mb but in my experience has performance issue.
  • security and CPU power: the ciphers used for encryption matter and need some CPU power.

Personally I have plenty of IPSEC tunnels, mostly VTI with ospf running atop and it works very well, but for a new setup I would seriously consider wireguard; although I don’t have much experience with it yet.

The routers are not behind nat,. As for dynamic routing, i’ll use it if needed but there again, i don’t have much experience. I’ll want something that will scale up to more locations if needed.
CPU shouldn’t be an issue, we’re planning to run vyos on some supermicro firewall appliance servers with quad core xeon processors.
For me it’s a tossup between ipsec with ospf routing for a mesh type setup or wireguard. Maybe dmvpn? Does it work?

If you want something that scale then dynamic routing compatibility is really mandatory.

DMVPN should work, and according to this thread:
https://forum.vyos.io/t/can-wireguard-replace-dmvpn/3219

could be a better choice than WireGuard for mesh deployment, but since I have no experience with it I will let more knowledgeable people answer.

I’d just like to put a plug in for OpenVPN. I have several OpenVPNs spanning 4 continents and they run for long uptimes without any issue, reconnecting over and around network issues and glitches repeatedly and reliably. I commonly send large backups over OpenVPN which sustain 40-50megabit for hours. I have used OpenVPN bridge-tunnels to send dual stack (ipv4 and ipv6), multiple vlans, and even jumbo frames through to the other side without issue, again reliably, for years. vrrp-based redundancy - check. Dynamic routing protocols - check. Multiple NATs - check. VPNs from dynamic-to-dynamic addresses using dyndns resolution - check. I don’t think I’ve failed a single use-case yet with OpenVPN. And I find vyos does it better than any other package out there.

We have been using OpenVPN for quite some time for about 20 sites around the globe (china, brazil, india, germany, sweden, UK, etc). A few months back we started to switch over to Wireguard and I will never look back. Stability on sites with “shaky” internet connections (Brazil & China) is a lot better. Performance is a lot better and CPU utilization a lot lower.

For some sites we initially had some quite “weird” problems (not being able to reach some web-sites, etc). They seem to be because path MTU discovery does not always work and we added the following option for the wireguard interfaces:
set firewall options interface wg01 adjust-mss '1380’

The configuration of wireguard tunnels is also extremely uncomplicated.

1 Like

How are you handling routing?
I assume site 2 and site 3 can talk directly to each other without going through site 1?
Did how routing is done change when you switched from openvpn to wireguard?

My gut feeling is to use wireguard, but it’s still new enough i wasn’t sure if i should or not.

Thanks for all the thoughts so far.

I have tunnels between all sites that have close interactions with each other (where there are users working together in different sites or servers that need to talk to each other).

For more “seldom” inter-site communication they will have to be routed over the HQ site.

I have changed from OSPF based routing to only use static interface routes when I switched to WireGuard.

Why did you dump OSPF?

Thanks.

The network setup is quite static and each site only has a default route back to HQ over the tunnel and for the site interconnect tunnels is was also quite simple setups in respect to routing.

There were no excuse to complicate things more than necessary…