Balancing two providers, one with extra AS

Hello, I have two routers each with a 10 gig upstream IP connection. router1 upstream is Phoenix NAP with AS19437 AS12189 then their peers and router2 upstream is Crown Castle with simply AS46887 then their peers. Because of this by default all traffic comes in/out Crown Castle even tho if I ignored AS19347 Phoenix NAP would have many better routes. I thought the best way to fix this was to prepend my AS398334 on inbound Crown Castle routes to even out the effect of Phoenix NAP extra AS19437. When I did that the routes on router2 looked balanced:

vyos@router2:~$ show ip bgp sum

IPv4 Unicast Summary:
BGP router identifier 204.117.64.3, local AS number 398334 vrf-id 0
BGP table version 6423887
RIB entries 1480608, using 260 MiB of memory
Peers 9, using 184 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
144.121.130.193 4      46887  398600  133525        0    0    0 00:26:04       806324
204.89.189.2    4     398334  758375  811717        0    0    0 17:06:43       781774

However, router1 only took 5 routes from router2 so all traffic still went out router1, the default router.

vyos@router1:~$ show ip bgp sum

IPv4 Unicast Summary:
BGP router identifier 204.117.64.2, local AS number 398334 vrf-id 0
BGP table version 5654228
RIB entries 1480537, using 260 MiB of memory
Peers 8, using 164 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
172.24.255.14   4      64527  402833    1035        0    0    0 17:08:22       781731
204.89.189.3    4     398334  687195  619756        0    0    0 17:08:04            5

My config is:

router1:

policy {
    prefix-list bogons {
        rule 10 {
            action permit
            description "this network [RFC1122]"
            le 32
            prefix 0.0.0.0/8
        }
        rule 11 {
            action permit
            description "private space [RFC1918]"
            le 32
            prefix 10.0.0.0/8
        }
        rule 12 {
            action permit
            description "CGN shared [RFC6598]"
            le 32
            prefix 100.64.0.0/10
        }
        rule 13 {
            action permit
            description "localhost [RFC1122]"
            le 32
            prefix 127.0.0.0/8
        }
        rule 14 {
            action permit
            description "link local [RFC3927]"
            le 32
            prefix 169.254.0.0/16
        }
        rule 15 {
            action permit
            description "private space [RFC1918]"
            le 32
            prefix 172.16.0.0/12
        }
        rule 16 {
            action permit
            description "TEST-NET-1 [RFC5737]"
            le 32
            prefix 192.0.2.0/24
        }
        rule 17 {
            action permit
            description "6to4 anycast relay [RFC7526]"
            le 32
            prefix 192.88.99.0/24
        }
        rule 18 {
            action permit
            description "private space [RFC1918]"
            le 32
            prefix 192.168.0.0/16
        }
        rule 19 {
            action permit
            description "benchmarking [RFC2544]"
            le 32
            prefix 198.18.0.0/15
        }
        rule 20 {
            action permit
            description "TEST-NET-2 [RFC5737]"
            le 32
            prefix 198.51.100.0/24
        }
        rule 21 {
            action permit
            description "TEST-NET-3 [RFC5737]"
            le 32
            prefix 203.0.113.0/24
        }
        rule 22 {
            action permit
            description multicast
            le 32
            prefix 224.0.0.0/4
        }
        rule 23 {
            action permit
            description "reserved for future use"
            le 32
            prefix 240.0.0.0/4
        }
    }
    prefix-list default {
        rule 10 {
            action permit
            prefix 0.0.0.0/0
        }
    }
    prefix-list local {
        rule 10 {
            action permit
            prefix 204.89.189.0/24
        }
        rule 11 {
            action permit
            prefix 204.117.64.0/24
        }
    }
    route-map external-in {
        rule 10 {
            action deny
            match {
                ip {
                    address {
                        prefix-list bogons
                    }
                }
            }
        }
        rule 11 {
            action deny
            match {
                ip {
                    address {
                        prefix-list default
                    }
                }
            }
        }
        rule 20 {
            action permit
        }
    }
    route-map local-out {
        rule 10 {
            action permit
            match {
                ip {
                    address {
                        prefix-list local
                    }
                }
            }
        }
    }
}
protocols {
    bgp 398334 {
        address-family {
            ipv4-unicast {
                network 204.89.189.0/24 {
                }
                network 204.117.64.0/24 {
                }
                redistribute {
                    connected {
                    }
                }
            }
        }
        neighbor 172.24.255.14 {
            address-family {
                ipv4-unicast {
                    route-map {
                        export local-out
                        import external-in
                    }
                    soft-reconfiguration {
                        inbound
                    }
                }
            }
            password ****************
            remote-as 64527
        }

router2:

policy {
    prefix-list bogons {
        rule 10 {
            action permit
            description "this network [RFC1122]"
            le 32
            prefix 0.0.0.0/8
        }
        rule 11 {
            action permit
            description "private space [RFC1918]"
            le 32
            prefix 10.0.0.0/8
        }
        rule 12 {
            action permit
            description "CGN shared [RFC6598]"
            le 32
            prefix 100.64.0.0/10
        }
        rule 13 {
            action permit
            description "localhost [RFC1122]"
            le 32
            prefix 127.0.0.0/8
        }
        rule 14 {
            action permit
            description "link local [RFC3927]"
            le 32
            prefix 169.254.0.0/16
        }
        rule 15 {
            action permit
            description "private space [RFC1918]"
            le 32
            prefix 172.16.0.0/12
        }
        rule 16 {
            action permit
            description "TEST-NET-1 [RFC5737]"
            le 32
            prefix 192.0.2.0/24
        }
        rule 17 {
            action permit
            description "6to4 anycast relay [RFC7526]"
            le 32
            prefix 192.88.99.0/24
        }
        rule 18 {
            action permit
            description "private space [RFC1918]"
            le 32
            prefix 192.168.0.0/16
        }
        rule 19 {
            action permit
            description "benchmarking [RFC2544]"
            le 32
            prefix 198.18.0.0/15
        }
        rule 20 {
            action permit
            description "TEST-NET-2 [RFC5737]"
            le 32
            prefix 198.51.100.0/24
        }
        rule 21 {
            action permit
            description "TEST-NET-3 [RFC5737]"
            le 32
            prefix 203.0.113.0/24
        }
        rule 22 {
            action permit
            description multicast
            le 32
            prefix 224.0.0.0/4
        }
        rule 23 {
            action permit
            description "reserved for future use"
            le 32
            prefix 240.0.0.0/4
        }
    }
    prefix-list default {
        rule 10 {
            action permit
            prefix 0.0.0.0/0
        }
    }
    prefix-list local {
        rule 10 {
            action permit
            prefix 204.89.189.0/24
        }
        rule 11 {
            action permit
            prefix 204.117.64.0/24
        }
    }
    route-map external-in {
        rule 10 {
            action deny
            match {
                ip {
                    address {
                        prefix-list bogons
                    }
                }
            }
        }
        rule 11 {
            action deny
            match {
                ip {
                    address {
                        prefix-list default
                    }
                }
            }
        }
        rule 20 {
            action permit
            set {
                as-path-prepend 398334
            }
        }
    }
    route-map local-out {
        rule 10 {
            action permit
            match {
                ip {
                    address {
                        prefix-list local
                    }
                }
            }
        }
    }
}
protocols {
    bgp 398334 {
        address-family {
            ipv4-unicast {
                network 204.89.189.0/24 {
                }
                network 204.117.64.0/24 {
                }
                redistribute {
                    connected {
                    }
                }
            }
        }
        neighbor 144.121.130.193 {
            address-family {
                ipv4-unicast {
                    route-map {
                        export local-out
                        import external-in
                    }
                    soft-reconfiguration {
                        inbound
                    }
                }
            }
            password ****************
            remote-as 46887
        }

Also, this does nothing to fix the problem that all traffic comes in Crown Castle.

Hi @sipvoip
You can’t use prepend for import, only for export. It’s standard of protocol BGP.
For import, you can use “local-pref” and “weight”

On R1 use parameter next-ho-self to your ibgp neighbor
set protocols bgp 398334 neighbor 204.89.189.2 address-family ipv4-unicast nexthop-self
Otherwise, the routes may not fall into the routing table R2.

On R2 use local-pref on import.
An example:

set policy route-map IBGP-IN rule 10 action permit
set policy route-map IBGP-IN rule 10 set local-preference 150
set protocols bgp 398334 neighbor 204.89.189.3 address-family ipv4-unicast route-map import IBGP-IN

For first try only “next-hop-self” parameter and check the result.

Hmm, lots of places talk about inbound and outbound prepending such as: https://sites.google.com/site/amitsciscozone/home/bgp/bgp-as-path-prepending-and-as-path-filters

next-hop-self makes things cleaner, my config was working because I had redistribute connected and that made sure that each router could reach the other point to point /30 ISP interfaces.

The AS-PATH I get from router1 all have 64527:19437:12189 before you see the their transit ASNs. on router2 I just get 46887 then their transit ASNs. I fixed my inbound traffic by prepending 46887 once on the outbound routes, this gave me a balanced view inbound. For outbound I prepended 46887 twice (one extra time for the stupid private ASN). Changing the ASN from my own to my ISPs fixed the issue where routes from router2 were not getting into router1.

@sipvoip, Sorry, Yes it worked and for import routes, I check it in the test lab, if you set prepend as last AS
Good trick.
Did it solve all your questions or is there something else?

So what’s is the final config snippet to pre-pend your AS inbound?