Sizing for 50 Tunnels with BGP over IPsec

General questions
1

Hi Team,

I am going to configure 50 Site-Site tunnels with BGP over IPsec wondering below hardware will suffice the need? Can someone please confirm?

  • 16 GB RAM
  • 4 Core CPU i7
  • 500 GB HD
2

The number of tunnels are rarely a problem - the issue is how many routes will each neighbor feed you and what throughput and latency do you expect through these encrypted tunnels?

1 Like
3

So there are 25 sites and every location has 2 links i.e 25x2 = 50 and in that case every location will share the single subnet. They will make tunnels with HO and HO will share only 2 subnets with spoke

This is hub and spoke topology while at spoke end I have fortinet firewalls and Vyos I am going to install at HO

Latency is fine however connectivity is pretty important hence planning for BGP over IPsec and BFD.

4

Something to consider is if you can use wireguard instead of IPsec to get better performance?

Otherwise what have been an issue historically is that a single flow is often limited to the performance of a single core (that is multicore usage havent been enabled properly or configured by the admin).

For example with a 4 core system you might reach in total 1Gbps full duplex but for a single session that might just reach 250Mbps per direction.

Below link got some info even if its a few years old and compares performance on FreeBSD rather than Linux but still:

Things to consider is to test with various offloading settings for the NICs involved along with verifing if AES-NI is enabled and used.

Out of the blue I would expect a fairly modern 4 core i7 CPU be able to push at least 1Gbps full duplex (2Gbps in a single direction).

What will happen when it starts to lag behind is that latency will increase and then performance will drop. So when using BFD make sure you have enough of headroom so you dont end up with “false-positives” that incorrectly brings down your BGP peering.

Default with many vendors are in the range of 3x250ms or 3x300ms. In your case you might want to increase that into 3x500ms or such.

3 Likes
5

Thanks and appreciate your feedback that was really helpful