I have been advertise prefix from 1 ISP successfully. Currently can use my IP to VPN with several users too.
But now I’m try to advertise the same prefix to the second ISP from different Vyos router (different location) as below screenshot diagram.
Vyos 1 - have vpn to company A and company B using ip 103.x.x.1/24
My goal is if Vyos 1 is down, all VPN users will automatically change/move to Vyos 2.
ISP A is from europe and ISP B is from asia.
Currently I’m using prepend to make Vyos 2 as a secondary.
When I start to advertise 103.x.x.x/24 from Vyos 2 to ISP B, I still can ping 103.x.x.1 which is good and VPN to company A is still up. Traceroute to 103.x.x.1 from Company A its going to ISP A which is also good.
But the issue is my VPN to Company B is down. Traceroute to 103.x.x.1 from Company B shows that its going to ISP B instead of ISP A. When checking with ISP of Company B, ISP B is a close AS neighbor with them.
Why is this happen? is prepend is not good enough to make Vyos 2 as a secondary route?
What can I do the fix this?
Can I use BGP med/metric instead of prepend?
If so, do i need to bring up iBGP between Vyos 1 and Vyos 2 to make this work?
Prepend isn’t a magical fix.
Some ISPs will use localpref to always prefer routes from one source over another (for example to always route to a local peer instead of an expensive transit provider) Look at the (Juniper) BGP path selection process to see what I mean.
The only real “fix” available to deploy is to advertise a more specific prefix via your ISP B link. Given you’re already advertising a /24 you probably can’t advertise anything more specific to them, they’ll reject it.
Anyway - this isn’t a Vyos problem - it’s a BGP path selection problem with Company B’s ISP selecting ISP B’s path as the better one, instead of ISP A. You’ll have to work with them to try and come up with a solution. You can also try and prepend many more times, that may help you. You really want to get a few “show ip bgp”'s from your ISPs though…
To choose prefer ISP suppose to be assign by our own router right?
So maybe I could setup with a different approach.
I plan to connect Vyos 1 and Vyos 2 by iBGP (VPN gre/vti) that I can advertise 103.x.x.x/24 at both router with med/metric difference. This way I don’t have to ask ISP to manipulate route as it will make it complicated.
I need to advertise same prefix at both ISP as my goal is all Companies VPN to Vyos router via ip 103.x.x.1 and if Vyos 1 is down 103.x.x.1 will “appear” in Vyos 2 and all VPNs will still up via Vyos 2.
To chose how you traffic egresses (leaves) your network, yes, that’s entirely up to you.
To chose how your traffic ingresses (comes in) your network, is much harder as I’ve explained above. MED won’t help you with how traffic ingresses, it’s only considered if the same route is seen at two different places within the same AS, which ISP A and ISP B won’t be.
But yes, you should be able to advertise the prefix out both paths and, regardless of which router the traffic comes in, it should still be able to reach its destination properly. If you were advertising your VPN’s subnet out router B, but router B couldn’t actually reach your VPN, that would have broken things.
If you advertise 103.x.x.x/24 out both routers, but you only have 103.x.x.1/32 (for example) as your VPN endpoint on Vyos1, then even if traffic comes into Vyos2, you can have routing between V1/V2 (your IGP, ISIS or OSPF) and traffic will come into V2 but hop over to V1 to terminate. You would need some form of physical link between V1 and V2 and you haven’t shown that on the diagram, but I am making the assumption you have one.
Sure, this won’t work if V1 fails totally, as the VPN endpoint has gone away, but as long as both routers are up, it shouldn’t matter if traffic egresses V1 and ingresses V2, as long as it ends up at the correct place.
The fact that the routing is asymmetric won’t matter (unless you have firewalls somewhere, but not shown on the diagram, they won’t like asymmetric traffic) as all traffic still ends up at the right place.
Its a good idea. But my objective to advertise 103.x.x.1 at both routers. Both have same VPN configuration to Company A and B(VPN to V1 up while VPN to V2 as standby mode). So if VPN to V1 down, VPN will automatically up to V2 as VPN configuration with same 103.x.x.1 source ip.
But I think this is not possible as I cannot control/manipulate ingress traffic from 2 different ISP and ASN. Can work with same ISP BGP ASN with prepend control (already test and working)
