BGP + BFD config question

Can someone clear something up for me? There are 2 apparent locations where you can set BFD.

#1 set protocols bfd peer … source address …
#2 set protocols bgp … neighbor … bfd

I have a VyOS router running 1.3.2 and I only have it set in #1 location and it shows a BFD session with the other end of the BGP link, which is an EdgeOS router in this case. Is that all I need or do I need to also set bfd on the BGP neighbor?

Thanks

If you want to shorten BGP convergence time (i.e. fast detection of neighbor down) - you have to configure #2 also. Or it will use default BGP timers for peer down detection.

BFD session just means that other side talks BFD.

HTH

Thank you, I will configure both settings then.

So I gave that a try tonight, but it seems that is not working correctly. With BFD set only as #1 ‘show bfd peers brief’ shows the active peers with “up” status.

If I add #2 also, I get a duplicate set of BFD peers that are not up. See the following screenshot, this router has 3 BFD peers.

image

If I view the details on one peer address, I get 2 results as follows:

That’s kind of strange. First things first - why this version of VyOS? It’s pretty old. One of the possible causes - outdated FRRouting and/or bug.

Here’s my config snippet (that’s from R1, R2 is symmetrical):

protocols {
    bfd {
        peer 10.10.10.2 {
        }
    }
    bgp {
        neighbor 10.10.10.2 {
            address-family {
                ipv4-unicast {
                }
            }
            bfd {
            }
            remote-as 65000
        }
        system-as 65000
    }
}

Everithing works as expected:

vyos@R1:~$ show bfd peers brief 
Session count: 1
SessionId  LocalAddress                             PeerAddress                             Status         
=========  ============                             ===========                             ======         
3951150886 0.0.0.0                                  10.10.10.2                              up             
vyos@R1:~$ show bgp summary 

IPv4 Unicast Summary (VRF default):
BGP router identifier 192.168.1.16, local AS number 65000 vrf-id 0
BGP table version 0
RIB entries 0, using 0 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.10.10.2      4      65000         7         7        0    0    0 00:04:24            0        0 N/A

Total number of neighbors 1
vyos@R1:~$

BTW, could you pls attach your full configs? That’s the first thing to do actually if you want your question answered :wink:

Another idea - source address in BFD peer config. I don’t think it’s necessary (can’t check on 1.3.2 myself as I’m using rolling releases, tested on 1.4-rolling-202308220020).

We’re on 1.3.2, I didn’t think it was that old. We pulled 1.3.3 but had some issues compiling it so stuck with 1.3.2 for now. These are tower routers in production, not a casual thing to just upgrade them all remotely.

Here is the full config from the router:

 interfaces {
     ethernet eth0 {
         hw-id 00:90:0b:47:0b:c0
     }
     ethernet eth1 {
         address 10.2.18.1/29
         description packetflux
         hw-id 00:90:0b:47:0b:c1
     }
     ethernet eth2 {
         hw-id 00:90:0b:47:0b:c2
     }
     ethernet eth3 {
         description howard-s1
         hw-id 00:90:0b:47:0b:c3
         vif 100 {
             address 10.1.18.2/24
         }
         vif 101 {
             address 10.10.18.41/29
             description howard-018-r1
         }
         vif 204 {
             address 10.10.14.12/29
             description "sunset 5 VLAN 204"
         }
         vif 208 {
             address 10.10.18.3/29
             description "ordbend 60 VLAN 208"
         }
         vif 209 {
             address 10.10.18.11/29
             description "ordbend 5 VLAN 209"
         }
         vif 210 {
             address 10.10.50.20/29
             description "glenn 60 VLAN 210"
         }
         vif 211 {
             address 10.10.18.19/29
             description "david 60 VLAN 211"
         }
         vif 212 {
             address 10.10.18.27/29
             description "david 5 VLAN 212"
         }
         vif 213 {
             address 10.10.18.33/29
             description "simson 5 VLAN 213"
         }
     }
     ethernet eth4 {
         hw-id 00:90:0b:47:0b:c4
     }
     ethernet eth5 {
         hw-id 00:90:0b:47:0b:c5
     }
     loopback lo {
         address 10.254.18.2/32
     }
 }
 policy {
     route-map SIMSON-EXPORT {
         description "route map to Sam Simson tower"
         rule 10 {
             action permit
             set {
                 as-path-prepend "4200018002 4200018002 4200018002 4200018002 4200018002"
             }
         }
     }
 }
 protocols {
     bfd {
         peer 10.10.18.4 {
             multihop
             source {
                 address 10.10.18.3
             }
         }
         peer 10.10.18.20 {
             multihop
             source {
                 address 10.10.18.19
             }
         }
         peer 10.10.50.19 {
             multihop
             source {
                 address 10.10.50.20
             }
         }
     }
     bgp 4200018002 {
         address-family {
             ipv4-unicast {
                 network 10.2.18.0/29 {
                 }
                 network 10.10.14.8/29 {
                 }
                 network 10.10.18.0/29 {
                 }
                 network 10.10.18.8/29 {
                 }
                 network 10.10.18.16/29 {
                 }
                 network 10.10.18.24/29 {
                 }
                 network 10.10.18.40/29 {
                 }
                 network 10.10.50.16/29 {
                 }
                 network 10.254.18.2/32 {
                 }
             }
         }
         neighbor 10.10.14.11 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             remote-as 4200014002
         }
         neighbor 10.10.18.4 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             bfd {
             }
             remote-as 4200027001
         }
         neighbor 10.10.18.12 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             remote-as 4200027001
         }
         neighbor 10.10.18.20 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             bfd {
             }
             remote-as 4200001001
         }
         neighbor 10.10.18.28 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             remote-as 4200001001
         }
         neighbor 10.10.18.34 {
             address-family {
                 ipv4-unicast {
                     route-map {
                         export SIMSON-EXPORT
                     }
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             remote-as 4200028001
         }
         neighbor 10.10.18.42 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             remote-as 4200018001
         }
         neighbor 10.10.50.19 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             bfd {
             }
             remote-as 4200050001
         }
     }
 }
 service {
     monitoring {
         telegraf {
             prometheus-client {
                 allow-from 10.1.254.0/24
                 authentication {
                     password ****************
                     username *****************
                 }
                 listen-address 10.1.20.33
                 port 9273
             }
         }
     }
     snmp {
         community ************* {
         }
         contact support@streamitnet.com
     }
     ssh {
         port 22
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     conntrack {
         modules {
             ftp
             h323
             nfs
             pptp
             sip
             sqlnet
             tftp
         }
     }
     console {
         device ttyS0 {
             speed 115200
         }
     }
     flow-accounting {
         enable-egress
         interface eth3
         netflow {
             engine-id 100
             server 12.220.9.198 {
                 port 2055
             }
             version 9
         }
     }
     host-name howard-018-r2.streamitnet.com
     login {
         user ******* {
             authentication {
                 encrypted-password ****************************
             }
         }
     }
     name-server 8.8.8.8
     name-server 8.8.4.4
     ntp {
         server time1.vyos.net {
         }
         server time2.vyos.net {
         }
         server time3.vyos.net {
         }
     }
     syslog {
         global {
             facility all {
                 level info
             }
             facility protocols {
                 level debug
             }
         }
     }
 }

Why use multihop if your BFD peers are directly connected? And I’d try to remove source address from BFD peers also (see my snippet - it’s pretty straightforward and copy pasted from documentation basically).

HTH

Multihop is used because theoretically it should work on 1 or many hops, you don’t have to change the config.

I just tried removing the multihop part and it completely drops that peer out of the ‘show bfd peers brief’ output. If I put multihop back then it brings it right back up again.

Removed the source address, no change, it’s down when multihop is gone, up when it’s there.

Looking at “how other does it” the below is from Arista when it comes to BFD:

For BFD to function as a failure detection mechanism, it must be enabled for each participating protocol.

So converting the above into VyOS Im not sure what the global BFD setting would do in VyOS (other than just some statistics that “yup - the neighbor replies”).

My guess is that you define the peers in the global setting (perhaps with ability to have custom timers if wanted per neighbor) and then enable BFD for each neighbor (if we talk about BGP) where you want it to exist?

On the other hand that sounds a bit off since the below example is how you do it on an Arista box:

switch(config)# router bgp 300
switch(config-router-bgp)# neighbor 10.13.64.1 bfd
switch(config-router-bgp)#

That is the only global setting when it comes to BFD which Arista have is the timers, everything else is configured at protocol level (per neighbor).

router bfd
   interval 300 min-rx 300 multiplier 3 default
!
1 Like

Thanks, I do agree looking at how other platforms do it is usually a helpful thing. EdgeOS seems to be happy just setting it up on the BGP neighbor config

set protocols bgp 1234 neighbor 10.10.18.3 fall-over bfd

And before we dropped OSPF it worked as well in the set protocols OSPF config.

I have the following on VyOS:

set protocols bfd profile BGP interval multiplier '3'
set protocols bfd profile BGP interval receive '300'
set protocols bfd profile BGP interval transmit '300'
set protocols bgp peer-group v4-transit bfd profile 'BGP'

Where peer-group could also be neighbor.

(ah sorry, this is for 1.4/1.5… missed you are on 1.3)

@roedie could you pls share ‘show bfd peer …’ output? I guess peer type should be dynamic in this case.

TIA

I think here’s some blind spot in documentation :frowning: As I can understand from my tests - when you configure protocol bfd you can tune timers (or you can use profile as @roedie suggested). After that, when you enable bfd in BGP neighbor VyOS tries to match static BFD peer (configured in protocol bfd clause) to this particular BGP neighbor.

But here’s another config I’ve tested:

vyos@R1# show protocols 
 bgp {
     neighbor 10.10.10.2 {
         address-family {
             ipv4-unicast {
             }
         }
         bfd {
         }
         remote-as 65000
     }
     system-as 65000
 }
[edit]

And looks like it works pretty well (pls note peer type):

vyos@R1# run show bfd peers 
BFD Peers:
        peer 10.10.10.2 vrf default
                ID: 449741054
                Remote ID: 4279051765
                Active mode
                Status: up
                Uptime: 8 minute(s), 47 second(s)
                Diagnostics: ok
                Remote diagnostics: ok
                Peer Type: dynamic
                RTT min/avg/max: 0/0/0 usec
                Local timers:
                        Detect-multiplier: 3
                        Receive interval: 300ms
                        Transmission interval: 300ms
                        Echo receive interval: 50ms
                        Echo transmission interval: disabled
                Remote timers:
                        Detect-multiplier: 3
                        Receive interval: 300ms
                        Transmission interval: 300ms
                        Echo receive interval: 50ms

[edit]

So VyOS BFD documentation defenitely needs some updates :slight_smile:

Here’s the output:

        peer x.x.x.x local-address x.x.x.y vrf default interface ethx
                ID: 2812836918
                Remote ID: 1380
                Active mode
                Status: up
                Uptime: 16 day(s), 3 hour(s), 13 minute(s), 6 second(s)
                Diagnostics: ok
                Remote diagnostics: ok
                Peer Type: dynamic
                RTT min/avg/max: 0/0/0 usec
                Local timers:
                        Detect-multiplier: 3
                        Receive interval: 300ms
                        Transmission interval: 300ms
                        Echo receive interval: 50ms
                        Echo transmission interval: disabled
                Remote timers:
                        Detect-multiplier: 3
                        Receive interval: 1000ms
                        Transmission interval: 1000ms
                        Echo receive interval: disabled

In FRR there are 2 types of BFD configuration, static and dynamic. I believe those translate through to VyOS BFD being configured inside BGP/OSPF/etc and on it’s at, I.E. at protocols → bfd

Which makes sense to me because only my BGP configured BFD comes up and creates a pair, because the other end is an EdgeOS router expecting a Dynamic BFD peer. So this pretty much tell me 99% that all we need is to configure BFD at protocols → bgp/ospf and it will bring up a proper session. protocols → bfd should be left alone unless you are building a specific situation.

So basically it depends on the remote peer configuration (EdgeOS here).

Am I getting this right - when you configure BFD in BGP only (no protocol bfd) everything work like expected?

Yes, I am 99% certain that is correct, after reading the FRR docs.

set protocol bfd... allows you to manually configure the details. set protocol bgp 1234 neighbor 1.2.3.4 bfd... enables dynamic BFD where it tries to figure out what the other side wants and matches it.

Looking at “how others does it” over at Arista:

BFD Modes
BFD functions in asynchronous or demand mode, and also offers an echo function. EOS supports asynchronous mode and the echo function.

- Asynchronous Mode
- Demand Mode

Asynchronous Mode
In asynchronous mode, BFD control packets are exchanged by neighboring systems at regular intervals. If a specified number of sequential packets are not received, BFD declares the session to be down.

Demand Mode
In demand mode, once the BFD session is established, the participating systems can request that BFD packets not be sent, then request an exchange of packets only when needed to verify connectivity. EOS does not support demand mode.

Echo Function
When the echo function is in use, echo packets are looped back through the hardware forwarding path of the neighbor system without involving the CPU. Failure is detected by an interruption in the stream of echoed packets. The minimum reception rate for BFD control packets from the neighbor is also changed automatically when the echo function is operational, because liveness detection is supplied by the echo packets.

While BFD control messages are transmitted to port 3784, BFD echo messages use UDP port 3785 for both source and destination.

I assume the “dynamic” mode in VyOS is what Arista referers to as “Demand Mode”?

Personally I think the dynamic mode is a bit sketchy when it comes to BFD.

To me using BFD is a way to speed up the detection if the a specific neighbor really is down or not (specially in a multihop scenario) without having to wait for some BGP timeout to kick in (which you normally dont want to be too low due to the load on the mgmt-cpu which then would occur). Which gives that asynchronous aka static mode is the prefered one in most cases.

That is I know ahead of time that within lets say 3x250=750ms BFD will detect that this path is no longer working and trigger BGP to remove that neighbor (way faster than to wait for a BGP timeout to trigger).