[Bug Reported] BGP routes marked inactive in routing table

bgp

#1

Hi,

I have setup a BGP peering on my VyOS 1.2.0-rolling+201811150337 VM and all of the BGP table shows all the received routes as “valid” and “best”.

BGP table version is 547, local router ID is 172.20.142.33, vrf id 0
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop’s vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*> 10.9.8.0/24 172.20.170.194 0 4242423934 4242420789 4242423759 i
*> 10.11.0.0/16 172.20.170.194 0 4242423934 4242423640 65043 64875 i
*> 10.11.160.0/20 172.20.170.194 0 4242423934 4242420119 65043 64875 i
*> 10.23.0.0/16 172.20.170.194 0 4242423934 4242420119 65043 65210 i
*> 10.26.64.0/18 172.20.170.194 0 4242423934 4242421926 4242420022 i
*> 10.26.96.0/20 172.20.170.194 0 4242423934 4242421926 4242420022
…continued

Those BGP routes are then also in the routing table but they all show up as inactive

Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route

S>* 0.0.0.0/0 [254/0] via 192.168.0.1, eth0, 00:13:24
B 10.9.8.0/24 [20/0] via 172.20.170.194 inactive, 00:13:20
B 10.11.0.0/16 [20/0] via 172.20.170.194 inactive, 00:13:20
B 10.11.160.0/20 [20/0] via 172.20.170.194 inactive, 00:13:20
B 10.23.0.0/16 [20/0] via 172.20.170.194 inactive, 00:13:20
… Continued

I can ping the next hop fine, but I can’t find any information about why they are failing to be added to the routing table properly. Anybody seen this kind of behaviour before and know how to fix it?

Thanks

Abby


#2

Okay so I seem to have found the problem in the wireguard tunnel configuration between my router and the BGP peer. Seems when setting up the tunnel using the VyOS command it did not set the ptp tunnel destination address correctly. Is there some command I missed or is this maybe a bug?

Abby


#3

It’s hard to find something out with your provided information, can you please share your wg commands you used?


#4

Sorry,

Here is the config for the wireguard tunnel

 wireguard wg01 {
        address 172.20.142.33/29
        address fdea:2eb5:7420::1/64
        description "uk-lon1"
        peer uk-lon1 {
            allowed-ips 0.0.0.0/0
            endpoint 142.93.XX.X:22673
            pubkey ****************
        }
        port 22673
    }

But I discovered that this setup setup sets the ptp tunnel destination address to the same as the local IP address 172.20.142.33 in this case (shown using sudo ifconfig) and using (sudo ifconfig wg01 dstaddr XXXX) and then putting the other end of the tunnels IP address in there solves the problem. I just can’t find any way in VyOS to do this command so that it will stick between reboots


#5

Don’t worry about your public key, there is a reason why it’s called public key :).
I assume ’ show interfaces’ shows you wg01 as up with an IP assigned?
If not, that needs to be fixed first.
You will need a global host route for 142.93.XX.X (set protocol static …) and your default gw would be then via wg01, if I understand your approach correctly.


#6

Oh that’s just how VyOS displays the public key when doing ‘show configuration’

Yes ‘show interfaces’ show wg01 as having an IP assigned

When configuring wireguard on any regular linux box the static route is not required as when you setup the ptp link you specify the destination address of the link. Though when looking at the wireguards own setup videos, maybe it’s because my IP and the other IP are not in the same subnet and that’s why it’s necessary to manually specify the destination address. Either way an option to set this from within the vyos configuration rather than having to use ifconfig or ip would be nice


#7

lol, see I never checked that via the cli commands, you got me.

So, if I understand you correctly, you want to secure all your bgp traffic via wireguard, right?


#8

I’m running BGP as part of the DN42 network, so got to establish point to point links since the routers aren’t physically connected. Hence the IPs are not on the same subnet


#9

Ok, gotcha. You need to set an interface route then.
https://wiki.vyos.net/wiki/Wireguard


#10

Yeah that’s what I originally had but BGP didn’t like that any of the routes being advertised by my peers. However when I removed that static route and set the destination address using ifconfig it works perfectly


#11

do you have ebgp-multihop set for your neighbor if it is not directly connected?


#12

No, the wireguard tunnel allows for them to be virtually connected over the single hop


#13

Yeah, but if you have on the other side a different network, you are not in the same subnet.


#14

Exactly, we’re not in the same subnet, so the wireguard tunnels endpoint are our public IPs so they can connect but then the wireguard IPs are not on the same subnet either, which works fine if the destination address is set to the remote ends wireguard IP but not if it’s set to the local IP which is the case as is currently setup by VyOS when creating the interface. The static route works for sending the traffic to the remote ends IP down the tunnel but not for anything else it seems


#15

allow-ips is just a filter method, it doesn’t handle any routing , it’s just matching IPs. Your endpoints are the addresses receive the encrypted payload and decrypt it. Once you set an interface route, your packet has src and dest ip, routed now via the wg interface. Allow-ips does a check if the destination address fits (you can have multiple one if you are not using 0.0.0.0/0), if it does it encrypts it with the configured public key of the peer and sends it via udp to the defined endpoint. So, in your first example your traffic used your default route using an eth interface.
If you define a static host route for the endpoints and have the default route via the wg interface, all traffic is being encrypted, except the traffic you need for your endpoints.


#16

Yeah I have Allow-IPs set to 0.0.0.0/0 that’s not a problem.

My thing is though that the static route shouldn’t be neccesary if the ptp is setup as I described with the destination address set as the remote wireguard IP. It’s not all traffic that is meant to be sent over the wireguard link, since I now have multiple of them. Just the traffic for that BGP peer or the traffic being sent over routes advertised by that peer. All the routes sent by BGP were showing up as inactive using the static route method but works fine using the destination address setting


#17

Hey there,

so through numerous different configurations of this I still can’t get the routes to not be “inactive” in the routing table, despite the BGP table saying they’re fine so I’m hoping that the current config and outputs might be helpful to somebody who might spot something I haven’t. Both wireguard tunnels are showing succesful handshakes so they look like they’re fine, the BGP sessions are established and sharing routes and those routes are being put into the BGP routing table as shown below, but then they either make it into the routing table but are marked inactive (ipv4) or don’t make it into the routing table at all (ipv6)

Current Config:

interfaces {
    ethernet eth0 {
        address 192.168.0.250/24
        duplex auto
        hw-id 52:54:00:a2:1e:91
        smp-affinity auto
        speed auto
    }
    ethernet eth1 {
        duplex auto
        hw-id 52:54:00:2c:d9:eb
        smp-affinity auto
        speed auto
    }
    loopback lo {
    }
    wireguard wg01 {
        address 172.20.142.33
        peer uk-lon1 {
            allowed-ips ::/0
            allowed-ips 0.0.0.0/0
            endpoint 142.93.XX.X:22673
            pubkey ****************
        }
        port 22673
    }
    wireguard wg02 {
        address 172.20.142.33
        description "uk-Lon1"
        peer burble {
            allowed-ips 0.0.0.0/0
            endpoint 185.121.XX.XX:31743
            pubkey ****************
        }
        port 31743
    }
}
policy {
    prefix-list BGP-OUT {
        rule 10 {
            action permit
            prefix 172.20.142.32/29
        }
    }
    prefix-list6 BGP6-OUT {
        rule 10 {
            action permit
            prefix fdea:2eb5:7420::/48
        }
        rule 20 {
            action deny
            prefix ::/0
        }
    }
    route-map BGP-OUT {
        rule 10 {
            action permit
            match {
                ip {
                    address {
                        prefix-list BGP-OUT
                    }
                }
            }
        }
        rule 15 {
            action permit
            match {
                ipv6 {
                    address {
                        prefix-list BGP6-OUT
                    }
                }
            }
        }
        rule 20 {
            action deny
        }
    }
    route-map IPv6-BGP-NextHop-Fix {
        rule 1 {
            action permit
            match {
                ipv6 {
                    nexthop ::
                }
            }
        }
        rule 2 {
            action permit
            set {
                ipv6-next-hop {
                    local fe80::42:196
                }
            }
        }
    }
}
protocols {
    bgp 4242422673 {
        address-family {
            ipv4-unicast {
                network 172.20.142.32/29 {
                }
            }
            ipv6-unicast {
                network fdea:2eb5:7420::/64 {
                }
            }
        }
        neighbor 172.20.129.187 {
            address-family {
                ipv4-unicast {
                    prefix-list {
                        export BGP-OUT
                    }
                }
            }
            remote-as 4242422601
        }
        neighbor 172.20.170.194 {
            address-family {
                ipv4-unicast {
                    filter-list {
                    }
                    prefix-list {
                        export BGP-OUT
                    }
                }
            }
            description "Lon-1"
            remote-as 4242423934
        }
        neighbor fe80::42:194 {
            address-family {
                ipv4-unicast {
                    soft-reconfiguration {
                        inbound
                    }
                }
                ipv6-unicast {
                    route-map {
                        import IPv6-BGP-NextHop-Fix
                    }
                    soft-reconfiguration {
                        inbound
                    }
                }
            }
            interface wg01
            remote-as 4242423934
        }
        parameters {
            router-id 172.20.142.33
        }
    }
    static {
        interface-route 172.20.129.187/32 {
            next-hop-interface wg02 {
            }
        }
        interface-route 172.20.170.194/32 {
            next-hop-interface wg01 {
            }
        }
        interface-route6 ::/0 {
            next-hop-interface wg01 {
            }
        }
        interface-route6 fe80::42:194/128 {
            next-hop-interface wg01 {
            }
        }
        route 0.0.0.0/0 {
            next-hop 192.168.0.1 {
                distance 254
            }
        }
    }
}

IFCONFIG of wireguard interfaces

wg01      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:172.20.142.33  P-t-P:172.20.142.33  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP  MTU:1420  Metric:1
          RX packets:456 errors:0 dropped:188 overruns:0 frame:0
          TX packets:461 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:93528 (91.3 KiB)  TX bytes:40632 (39.6 KiB)

wg02      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:172.20.142.33  P-t-P:172.20.142.33  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP  MTU:1420  Metric:1
          RX packets:460 errors:0 dropped:0 overruns:0 frame:0
          TX packets:379 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:121540 (118.6 KiB)  TX bytes:34840 (34.0 KiB)

show ip bgp neighbors 172.20.129.187 routes

BGP table version is 671, local router ID is 172.20.142.33, vrf id 0
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*  10.9.8.0/24      172.20.129.187           0             0 4242422601 64719 4242423759 i
*  10.11.0.0/16     172.20.129.187           0             0 4242422601 4242423640 65043 64875 i
*  10.11.160.0/20   172.20.129.187           0             0 4242422601 4242423640 65043 64875 i
*  10.23.0.0/16     172.20.129.187           0             0 4242422601 4242423640 65043 65210 i

show ip bgp neighbors 172.20.170.194 routes

BGP table version is 671, local router ID is 172.20.142.33, vrf id 0
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.9.8.0/24      172.20.170.194                         0 4242423934 4242420789 4242423759 i
*> 10.11.0.0/16     172.20.170.194                         0 4242423934 4242423640 65043 64875 i
*> 10.11.160.0/20   172.20.170.194                         0 4242423934 4242421191 65043 64875 i
*> 10.23.0.0/16     172.20.170.194                         0 4242423934 4242421191 65043 65210 i
*> 10.26.64.0/18    172.20.170.194                         0 4242423934 4242421926 4242420022 i

show ip route

Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route

S>* 0.0.0.0/0 [254/0] via 192.168.0.1, eth0, 00:51:06
B   10.9.8.0/24 [20/0] via 172.20.170.194 inactive, 00:51:01
B   10.11.0.0/16 [20/0] via 172.20.170.194 inactive, 00:51:01
B   10.11.160.0/20 [20/0] via 172.20.170.194 inactive, 00:51:01
B   10.23.0.0/16 [20/0] via 172.20.170.194 inactive, 00:51:01

Any ideas for why this won’t work would be greately appreciated, or if more outputs are needed I can provide them too


#18

I’m about to lab it up right now…
Can you send me your bgp config and routes please.


#19

If you want we can open a pm in slack.


#20

I shall send you it on there then