What is the best method for my BGP routers to share external BGP routes to my OSPF routers?

General questions
, ,
1

What is the best method for my BGP routers to share external BGP routes to my OSPF routers ?

I have two IPv4 BGP routers. Each of my BGP routers have different upstream BGP peers.
Both of my two BGP routers do not know any BGP routes information from my other BGP router.
Both of my BGP routers know my OSPF routes ( but do not put anything into my OSPF routers ).

I have multiple OSPF routers. All of my OSPF routers know the live-IP routes for all other OSPF routers.
None of my OSPF routers know any BGP routes.
All of my OSPF routers have a single default route to only one or the other of my BGP routers.
All of my OSPF routers use static routes to the rest of my other routers.

Some of my other non-BGP and non-OSPF routers are used for CGN-NAT , live-IP-routers and Wans/Lans to my customers.

My question - What is the best method for my OSPF routers to be able to get to the Internet if one of my BGP routers dies or drops/looses the BGP tables from either external upstream BGP peer ?

  • Should I have both of my BGP routers redistribute full BGP routing tables to my OSPF routers ? This would then introduce all BGP routes twice into my OSPF routing table.

  • I wish to configure something that is a best path out from my OSPF routers to the Internet and some redundancy if/when one of my BGP routers has a problem.

  • is there a way for each of my two BGP routers to put a single default route into my OSPF network ?

** I included a simple network drawing of what I have.
Red lines are BGP routes only.
Green lines are OSPF routes only.
Black lines are static routes only.

I’m open for ideas (OSPF , iBGP , something else … )

Thank you for ideas
North Idaho Tom Jones

BGP-OSPF-StaticRoutes-Customers
BGP-OSPF-StaticRoutes-Customers1880Ă—1306 290 KB

2

Man, you are quite brave.

1 Like
3

e: *Should I have both of my BGP routers redistribute full BGP routing tables to my OSPF routers ?
Re: … Man, you are quite brave …

Well , I don’t think putting two full IPv4 BGP routing tables into my OSPF network is a good idea because of the added CPU overhead on my OSPF routers might then be off off-the-wall insane.

4

Are you learning a default from each of your upstream peers? Are these the only places you learn defaults from?

5

There are most likely multiple ways to solve this…

Most expensive one would be to inject all the BGP data into your OSPF and then watch your OSPF routers explode (unless they are beefy enough to deal with give or take 1M unique routes per upstream).

You probably want your BGP routers to auto-summarize before injecting into the OSPF to shrink the number of routes with the cost of processing power (mgmt-cpu) when doing so - in other terms it will take more time after a reboot to deal and forward all the routes.

The good thing with above is that using two different ISPs most likely means that there are routes avaiable through ISP1 which isnt available through ISP2 and vice versa. So “correct” BGP-router will be used on first attempt.

The auto-summary would also mean that a BGP router who doesnt see the full network will inject the parts as longest prefix so that will become the prefered exit from IGP point of view - which can both be a good or a bad thing.

Another solution would be to setup some kind of RR (Route Reflector) internally. You will end up with a single point of failure but this way you can better process all the routes and optimize them however you like and then inject them into your OSPF. For example letting your OSPF use default and then only inject the differences between your ISP’s.

A variant of above would of course also be to let your two BGP routers to exchange info among themselves (and route traffic that way).

This way if some packet end up at BGP-R1 but ISP1 (which is its uplink) doesnt have a route for the destination then BGP-R1 would push traffic to BGP-R2 which then have this destination (and if destination doesnt exist at all it would be dropped before being pushed to the other side).

The good thing with above would be that you at your OSPF level would “only” need to deal with default routes.

Either with ECMP or manually putting the left side using BGP-R1 as default with lowest metric and BGP-R2 with higher metric while the right side use BGP-R2 as prefered default with BGP-R1 as secondary option (Im thinking of the OSPF-routers connected to your green network in your drawing).

I dont think there is a holy grail to this scenario (please enlighten me if it does :slight_smile: )

Either you do it “properly” but then must use overly beefy routers to be able to deal with all the RIB and FIB.

That is the OSPF router will select a proper egress on its own.

Or you “cheatout” using defaults towards your BGP-routers and then let them figure out if they should use the local ISP as egress or send to that other BGP-router of yours (which then use its local ISP as egress).

That is there will be some traffic which selects the “incorrect” egress and must be moved between the BGP-routers before actually egressing (otherwise it wont reach its destination).

Personally Im more into using BGP also internally (this way you dont have to figure out where “area 0” really is as with OSPF).

You can do things like neighbor 192.0.2.1 local-as 65100 no-prepend replace-as to segment your internal network into various ASN and then “hide” this when peering externally.

More of that in: https://old.reddit.com/r/Arista/comments/1jme90d/how_to_replace_the_whole_bgp_path_with_my_own/

6

Each of my BGP two BGP routers has a single static default route to the peer it is peering with.
If I remember correctly , my BGP routers do not need a static default route because they each get full BGP routing tables. So , I am pretty sure I can drop the default route in my BGP routers and still have everything still work.

7

The easy way out is to just do (static) default to both your ISP’s which gives that you internally just do default in OSPF (with or without ECMP) and call it a day.

This gives that with just two BGP-routers all internal routers will at most have 2 routes (the default ones towards your BGP-routers).

Then when you figure out if a prefix only exists through ISP1 but not ISP2 you can put that manually in your OSPF (or let BGP-R1 announce it through OSPF). The longest prefix will win meaning that manual prefix will end up with the BGP-router of yours which announce it internally.

The purpose of doing BGP towards internet would then be a matter of announcing your presence on the internet (and by prepending letting the internet know which ISP your prefer for ingress traffic).

8

If that’s the case:

  1. Definitely remove the static default route. You want your downstreams to be aware of an upstream failure, so you want the default to be withdrawn in the case one of those peerings go down. Otherwise traffic could blackhole.
  2. Only advertise your default into OSPF using default-information originate. This will import just the default into OSPF (the default has to be present in the RIB, so it’ll get withdrawn from OSPF if it is lost in the BGP table).
  3. You don’t need the full BGP table, since all paths lead towards the default anyways. You can selectively redistribute some routes into OSPF if you want them to be preferred out of one of the peerings (like AWS, Azure, etc…). You can do that with as-path lists.
  4. Depending on how redundant your cabling is between the BGP and OSPF routers, you may want to peer your 2 BGP routers to protect against a double failure (e.g. peering to remote1 is down, and peering to OSPF router 2 is down. Traffic could then flow from ospf, to BGP1, to BGP2, to remote2.
1 Like
9

Not only that, if you want to know details read this article

1 Like
10

Thank both of you ( L0crian & Apachez ) for your input and very prompt replies.

*e: If that’s the case:
*e: 4. Depending on how redundant your cabling is between the BGP and OSPF routers, you may want to peer your 2 BGP routers to protect against a double failure (e.g. peering to remote1 is down, and peering to OSPF router 2 is down. Traffic could then flow from ospf, to BGP1, to BGP2, to remote2.
Re: … A variant of above would of course also be to let your two BGP routers to exchange info among themselves (and route traffic that way). …

I suspect in my case , option 4 ( having both of my BGP routers exchange routes to/from each other ) might be what works best for me.

Both of my VyOS BGP routers are running on two different dedicated Proxmox servers with 40-Gig network cards and plenty of CPU/Ram.
However , some of my vm VyOS OSPF routers do not have the horse power to handle big additional loads.

So , now I just need to figure out how to get my two BGP routers to exchange routes or summarized routes.
Not sure where to start yet - read read read then ask some questions again.

Thank you

North Idaho Tom Jones

11

Are both of your BGP routers the same AS?

12

No - I have two different AS numbers.

13

Does this configuration added to both of my BGP routers work - so that if one BGP router does not have a BGP route - or any BGP routes , then default route to my other BGP router ?

To me , it looks like it would work ( as long as both of my BGP routers are up and at lease one of my BGP routers has BGP routes ). And to me , I would think it introduces very little/no CPU overhead.

BGP router #1
eth1 a.b.c.1/28 ( facing my OSPF network )
set protocols static router 0.0.0.0/0 next hop a.b.c.2 distance 10

BGP router #12
eth1 a.b.c.2/28 ( facing my OSPF network )
set protocols static router 0.0.0.0/0 next hop a.b.c.1 distance 10

I included a drawing with the config changes.

BGP-OSPF-StaticRoutes-Customers-Possible-Solution
BGP-OSPF-StaticRoutes-Customers-Possible-Solution1880Ă—1306 321 KB

14

Drawback with that is that you will have a routingloop if neither have an egress for a particular destination.

That will die out within 255 rounds but still.

So if your BGP-routers can fit multiple full internet tables that would be prefered so if BGP-R1 dont have a route towards ISP1 then it will also know if BGP-R2 do have that or not. And if not the packet is dropped right away and by that only affecting the green and black network once (well as long as the source continue to send packets but still).

15
  • Are these static routes towards your upstream peers?
    • If they are, you don’t want them. You want the defaults learned from upstream. This has an inherit healthcheck that your static routes do not. If your peering goes down, your static route would stay up and you could blackhole traffic.
  • If they are between your BGP routers:
    • You still don’t want it. You want to peer eBGP if they’re different ASNs. This dynamically ensures that an advertised default actually exists, preventing blackholing.

Considerations:

You want to ensure that your defaults that are learned via your upstream peerings are equal cost. Most defaults are directly advertised from your peer, so the as-path length should be 1, making both of them equal. When they’re advertised to your other BGP router, they’d pick up another as hop, making it less preferred locally. So you’d always use the local default, unless it is not available, in which case you’d use the path through the other router.

When you peer with BGP between your routers, you can advertise the full table, or just a default. If you can’t think of any reason why you need specific routes, just advertise the defaults.

16

Apachez , thanks for your input.

This is what I think I’m gonna do:

  • Short term ; quick dirty fix - have both BGP routers default route to the other ( yes possible routing loops until the counter hits 255 during ping-ping back and forth events ).
  • Long term ; Built an off-line LAB set of four BGP routers to simulate what I have , then configure my two BGP routers to share BGP routes. When I have a working configuration then dump the quick-fix and go with the long-term solution.

To simulate full BGP tables , I’m thinking about on my BGP routers, route print bgp then pipe it through sed and create BGP routes that looks similar to my two upstream BGP routers are giving me.

Or is this normally a simple set of commands on my BGP router to exchange BGP routes on my BGP routers ( I’m not strong on BGP - pretty weak actually …

17

If the static routes you showed were between your 2 BGP routers, don’t do that. That defeats the purpose of protecting against double failure, and as you have it, you will break your traffic. The distance of your eBGP learned routes will be 20, and you’re setting your static routes to lower than that, making them preferred.

The BGP config to peer your 2 routes is pretty basic. If we imagine your 2 ASNs are 1 and 2, it’d be like this:

######################
Router1:

set policy route-map Adv_DEFAULT rule 10 action 'permit'
set policy route-map Adv_DEFAULT rule 10 match ip address prefix-list 'DEFAULT'

set policy prefix-list DEFAULT rule 10 action 'permit'
set policy prefix-list DEFAULT rule 10 prefix '0.0.0.0/0'

set protocols bgp neighbor 10.1.2.2 address-family ipv4-unicast route-map export 'Adv_DEFAULT '
set protocols bgp neighbor 10.1.2.2 remote-as '2'

######################
Router2:

set policy route-map Adv_DEFAULT rule 10 action 'permit'
set policy route-map Adv_DEFAULT rule 10 match ip address prefix-list 'DEFAULT'

set policy prefix-list DEFAULT rule 10 action 'permit'
set policy prefix-list DEFAULT rule 10 prefix '0.0.0.0/0'

set protocols bgp neighbor 10.1.2.1 address-family ipv4-unicast route-map export 'Adv_DEFAULT '
set protocols bgp neighbor 10.1.2.1 remote-as '1'

This will peer between your 2 BGP routers, and advertise only their defaults between each other.

18

Another method is to define a high administrative distance for that default route and call it a day.

https://docs.vyos.io/en/latest/configuration/protocols/static.html

Something like:

set protocols static route 0.0.0.0/0 next-hop 192.0.2.1 distance 254

This way if a route is learned any other way (like BGP, OSPF etc) that learned route will be used instead.

Normally longest prefix have presence over lets say a default route (since lets say /24 is “longer” than /0) however the administrative distance is whats first being evaluated so a default (0.0.0.0/0) with distance 1 will win over a lets say 192.0.2.0/24 with distance 2 or higher (as I recall it).

Only when they have the same distance then the longest prefix (192.0.2.0/24 in this example) will win over the default of 0.0.0.0/0.

So for the shorterm solution I would probably add a static with highest administrative distance since the example L0crian gave will only work if the ISP sends a “0.0.0.0/0” towards your end - if they dont then you will lose using the BGP-routers as last resort for each other.

19

If you are learning full tables from your upstreams, why don’t you just generate a static holddown on your border routers?
set protocols static route 0.0.0.0/0 blackhole

Your two border routers learning the full BGP table from your upstreams have iBGP between them, so even if one upstream goes away the full table is still learnt from the neighbour.

And you can just redistribute your static 0.0.0.0/0’s into OSPF from both BGP routers, so all traffic (without a more specific in OSPF) will end up at the border router, which will then know the proper egress route via the learnt route from the full table. Should an upstream go away, the iBGP will be there to move traffic from the border router with the dead eBGP peer to the other border router with the still working eBGP peer.

Injecting a full table into OSPF isn’t workable - don’t do this :slight_smile:

20

For everyone; do not use a static route for this. It’s the wrong tool for the job and can lead to the exact failure scenario that is trying to be solved.

The reason you want routing between the 2 BGP routes is to route around failures. A static route is a hard coded route that can never fail, even if all other ability to forward on that router has ceased. And because the default is being imported into OSPF, a default will always be advertised from that router (since the static route is always up), even though it could be isolated from any upstream forwarding. OSPF would have no way of knowing.

If for some reason, this is the only AS that doesn’t have a default within their AS that they advertise to their downstream customers, then you want to originate the default with default-information originate in either BGP or OSPF, since you can do it conditionally and ensure traffic doesn’t get blackholed through bad design.