Configuration for fast BGP covergence

Hi Team,

I have two Vyos routers and have deployed in MPLS connections and configured two site-to-site vti tunnels with iBGP running so here is the scenario.

RouterA
eth0: 192.168.5.10
eth1: 172.16.1.10
vti5: 169.254.254.171
vti10: 169.170.170.171
AS65001
holdtime 6
keepalive 2
multi-path 2

RouterB
eth0: 10.10.11.10
eth1: 192.168.44.10
vti5: 169.254.254.170
vti10: 169.170.170.170
AS:65001
holdtime 6
keepalive 2
dpd action restart
multi-path 2

Now if I shutdown primary link. Tunnel goes down for about a minute or so and even if I other link is up ip bgp does not show the routes received. Almost after 1-2 minute BGP routes appears and the traffic fails over.

So any idea what should be the ideal configuration for fast convergence and minimal packet loss?

Wireguard + bfd + bgp

1 Like

Hmm…those are pretty new to me. Any existing config available or template would help me to understand much faster.
Any way I am researching on my own as well.

Hi @Viacheslav

Any idea if we can achieve multipath with wireguard+BFD+BGP? I have not actually started implementing it but creating a topology hence wanted to ensure if multiple links can be achieved using this solution?

TIA

Unfortunately, I don’t have the experience with VyOS to provide instructions, but BFD works in conjunction with a routing protocol to detect a path outage faster than a routing protocol normally would. For example, BGP may take 60sec to time out, while BFD might poll more quickly, like once every 100ms and take a path down after 5 missed bfd replies before BGP detects the outage. BFD doesn’t replace any routing protocol, it just works along side it to detect when a path has gone down quickly.

I think the wireguard suggestion, is just because it’s easier than IPSec. I’ve spent many moons troubleshooting IPSec and very little to show for it. Even after years I still get stumped by issues and get frustrated with it. Wireguard has a lot of positive attributes about it. It’s simpler to set up and operate, it deals with packet fragmentation well, and it performs well on standard hardware.

Hi @James.harr

Yes I figured that out and its really fast as compare to traditional BGP timeout. Manage to work with wireguard+BGP+BFD

You should have good result as your expectations. I have implemented wireguard+bgp+bfd to connect many sites and very satisfied with the result.

1 Like

What is the vyos version you used on your prod routers?

Could you share what you did so others might benefit for that?

Sure - Sharing is caring :smiley:

R1
From $mode 
generate wireguard keypair

#######################

set interfaces ethernet eth0 address '192.168.47.20/24'
set interfaces ethernet eth0 duplex 'auto'
set interfaces ethernet eth0 hw-id '00:0c:29:d4:6e:9e'
set interfaces ethernet eth0 smp-affinity 'auto'
set interfaces ethernet eth0 speed 'auto'
set interfaces ethernet eth1 address '10.30.30.20/24'
set interfaces ethernet eth1 duplex 'auto'
set interfaces ethernet eth1 hw-id '00:0c:29:d4:6e:a8'
set interfaces ethernet eth1 smp-affinity 'auto'
set interfaces ethernet eth1 speed 'auto'
set interfaces ethernet eth2 address '10.10.20.20/24'
set interfaces ethernet eth2 duplex 'auto'
set interfaces ethernet eth2 hw-id '00:0c:29:d4:6e:b2'
set interfaces ethernet eth2 smp-affinity 'auto'
set interfaces ethernet eth2 speed 'auto'
set interfaces loopback lo
set interfaces wireguard wg01 address '10.44.44.1/30'
set interfaces wireguard wg01 peer tun-wg02 allowed-ips '192.168.40.0/24'
set interfaces wireguard wg01 peer tun-wg02 allowed-ips '10.44.44.2/32'
set interfaces wireguard wg01 peer tun-wg02 endpoint '192.168.47.40:51820'
set interfaces wireguard wg01 peer tun-wg02 pubkey <COPY THAT PUBKEY>
set interfaces wireguard wg01 port '51820'
set interfaces wireguard wg02 address '10.88.88.1/30'
set interfaces wireguard wg02 peer tun-wg04 allowed-ips '192.168.40.0/24'
set interfaces wireguard wg02 peer tun-wg04 endpoint '10.30.30.40:51840'
set interfaces wireguard wg02 peer tun-wg04 pubkey  <COPY THAT PUBKEY>
set interfaces wireguard wg02 port '51840'
set protocols bgp 65001 address-family ipv4-unicast network 10.10.20.0/24
set protocols bgp 65001 neighbor 10.44.44.2 remote-as '65002'
set protocols bgp 65001 neighbor 10.44.44.2 update-source '10.44.44.1'
set protocols bgp 65001 timers holdtime '6'
set protocols bgp 65001 timers keepalive '2'
set protocols static
set service ssh
set system config-management commit-revisions '100'
set system console device ttyS0 speed '9600'
set system host-name 'R1'
set system login user vyos authentication plaintext-password ''
set system login user vyos level 'admin'
set system ntp server 0.pool.ntp.org
set system ntp server 1.pool.ntp.org
set system ntp server 2.pool.ntp.org
set system syslog global facility all level 'info'
set system syslog global facility protocols level 'debug'

AND R2
$ generate wireguard keypair

set interfaces ethernet eth0 address '192.168.47.40/24'
set interfaces ethernet eth0 duplex 'auto'
set interfaces ethernet eth0 hw-id '00:0c:29:b5:16:c7'
set interfaces ethernet eth0 smp-affinity 'auto'
set interfaces ethernet eth0 speed 'auto'
set interfaces ethernet eth1 address '192.168.40.40/24'
set interfaces ethernet eth1 duplex 'auto'
set interfaces ethernet eth1 hw-id '00:0c:29:b5:16:d1'
set interfaces ethernet eth1 smp-affinity 'auto'
set interfaces ethernet eth1 speed 'auto'
set interfaces ethernet eth2 address '10.30.30.40/24'
set interfaces ethernet eth2 duplex 'auto'
set interfaces ethernet eth2 hw-id '00:0c:29:b5:16:db'
set interfaces ethernet eth2 smp-affinity 'auto'
set interfaces ethernet eth2 speed 'auto'
set interfaces loopback lo
set interfaces wireguard wg01 address '10.44.44.2/30'
set interfaces wireguard wg01 peer tun-wg02 allowed-ips '10.44.44.1/32'
set interfaces wireguard wg01 peer tun-wg02 allowed-ips '10.10.20.0/24'
set interfaces wireguard wg01 peer tun-wg02 endpoint '192.168.47.20:51820'
set interfaces wireguard wg01 peer tun-wg02 pubkey  <COPY THAT PUBKEY>
set interfaces wireguard wg01 port '51820'
set interfaces wireguard wg02 address '10.88.88.2/30'
set interfaces wireguard wg02 peer tun-wg04 allowed-ips '10.10.20.0/24'
set interfaces wireguard wg02 peer tun-wg04 endpoint '10.30.30.20:51840'
set interfaces wireguard wg02 peer tun-wg04 pubkey  <COPY THAT PUBKEY>
set interfaces wireguard wg02 port '51840'
set protocols bgp 65002 address-family ipv4-unicast network 192.168.40.0/24
set protocols bgp 65002 neighbor 10.44.44.1 remote-as '65001'
set protocols bgp 65002 neighbor 10.44.44.1 update-source '10.44.44.2'
set protocols bgp 65002 timers holdtime '6'
set protocols bgp 65002 timers keepalive '2'
set protocols static
set service ssh
set system config-management commit-revisions '100'
set system console device ttyS0 speed '9600'
set system host-name 'R2'
set system login user vyos authentication encrypted-password '$6$Echv46oLnp5O/o$0JCnLzgSZ1j2LGFX9DL3Nx97IrfKDqNJbyDv8am3pU0kmQUzyzNtO3ZV/.Fd0klziYZ5h.zX6wZRGhrnIw5VT1'
set system login user vyos authentication plaintext-password ''
set system login user vyos level 'admin'
set system ntp server 0.pool.ntp.org
set system ntp server 1.pool.ntp.org
set system ntp server 2.pool.ntp.org
set system syslog global facility all level 'info'
set system syslog global facility protocols level 'debug'

Many thanks! Is the BFD setup additional to what you shared?

Simple example for bfd+bgp
R1 1.4 config

set protocols bfd peer 192.0.2.2 source address '192.0.2.1'
set protocols bgp local-as '65001'
set protocols bgp neighbor 192.0.2.2 bfd
set protocols bgp neighbor 192.0.2.2 remote-as '65002'

R2 1.2.8 config

set protocols bfd peer 192.0.2.1 source address '192.0.2.2'
set protocols bgp 65002 neighbor 192.0.2.1 bfd
set protocols bgp 65002 neighbor 192.0.2.1 remote-as '65001'

Poweroff or set interface down on R1 and check bgp state on R2:

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
192.0.2.1       4      65001       5       5        0    0    0 00:00:08       Active

Be attention you may need to increase bfd timers if RTT for remote peer so big.

1 Like

Correct - The bfd and bgp is then simple one