BGP preferred path

General questions
1

Hello,

I’m working in the following scenario over AWS:

3 VPCs:

  • 1 Coms VPC
  • N Production VPC
  • N Test or PoC VPC

The need is to intercunicate Production and Test VPC, with a low cost for Test and PoC ones.

On Coms VPC there are two VyOS instances (VyOS-TRN01 and VyOS-TRN02) connected to:

  • Direct Connect for OnPrem coms
  • Production VPCs through AWS VPN (2 VPN per Production VPC connected each one to one of the Coms VyOS)
  • Test VPCs through OpenVPN to another Vyos on that VPC (through VPC peering)

This scheme is working perfect for Productions ones (through AWS VPN), there is a small problem with BGP but by the moment we solved it through scripting.

A detail explanation of the conection between a Test VPC and the Coms VPC:

  • VyOS-Test1 (vtun0) connected to VyOS-TRN01 through openvpn (vtun0)
  • VyOS-Test1 (vtun1) connected to VyOS-TRN02 through openvpn (vtun1)
    These connections are working without problems, over them we configured BGP (AS 65000 for VyOS-TRN01/02 and AS 65001 for VyOS-Test1), Test1 receive all the routes with out problems.

Then we configure a static route in VyOS-Test1 for complete Test1 VPC (by default only have a route for the subnet where it is working):

  • 10.2.0.0/24 via 10.2.0.129 dev eth0 (set protocols static route 10.2.0.0/24 next-hop 10.2.0.129
    And announce it through bgp (set protocols bgp 65001 network 10.2.0.0/24)

Once it’s done we add the routes received from bgp in Test1 to the VPC Route Tables with target VyOS-Test1 instance, on this point if we do a ping (or ssh or whatever) from TestApp to ProdApp it works, but with tcpdump we see the following:

  • ICMP echo req goes through vtun0 (to TRN01) and from that point goes to ProdApp.
  • ICMP echo reply comes back through vtun1 (from TRN02), this is a problem for firewall because we lost the conntracking.

But if we try the same from one ProdVPC to another ProdVPC there is not that problem, the outgoing and incoming traffic travel over the same interface)

And now (after this long text) the question: how can i configure that TRN01 paths are the preferred? or avoid this problem?

Best regards

[attachment=190]