AWS VPC up, but no traffic

General questions
1

Hello,

I have an AWS VPC configuration, exported from the AWS Console. According the docs, I wrapped it in a script, removed the comments and updated the ipsec-interfaces.

The AWS console shows me the IPSec tunnel is up, also on the VyOS side, but I don’t get anything through. I also added IPSec firewall rules, and prevented NAT for what needs to go through the tunnel, but I’m still missing something.

The setup (see attached vyos config) is quite new. There are a number of interfaces defined, but only 2 are used at the moment: eth2 for Internet access (static IP, dhcp served) and my home lan on eth7 (static IP, dhcp server for lan machines). Currently, I’m only trying to ping an EC2 machine running in the VPC from my home lan.

Setup: VyOS 1.1.6 running on VMware ESXi 6.0

Can someone give me a clue on what I’m missing?

2

Hmm, one of the BGP neighbors doesn’t have any advertised-routes:

vyos@vyos:~$ show ip bgp neighbors 169.254.20.109 advertised-routes vyos@vyos:~$

The other one report this:

[code]vyos@vyos:~$ show ip bgp neighbors 169.254.20.89 advertised-routes
BGP table version is 0, local router ID is 172.16.50.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*> 10.128.0.0/16 169.254.20.90 0 9059 i

Total number of prefixes 1[/code]

BGP Summary:

[code]yos@vyos:~$ show ip bgp summary
BGP router identifier 172.16.50.254, local AS number 60000
IPv4 Unicast - max multipaths: ebgp 1 ibgp 1
RIB entries 1, using 96 bytes of memory
Peers 2, using 9120 bytes of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
169.254.20.89 4 9059 4688 4691 0 0 0 12:22:11 1
169.254.20.109 4 9059 4688 4689 0 0 0 12:22:15 1

Total number of neighbors 2[/code]

VTI interfaces:

[code]vyos@vyos:~$ show interfaces vti
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description

vti0 169.254.20.110/30 u/u VPC tunnel 1
vti1 169.254.20.90/30 u/u VPC tunnel 2[/code]

And netstat shows me it routes over the tunnel without any advertised-routes:

vyos@vyos:~$ netstat -nr Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 81.82.192.1 0.0.0.0 UG 0 0 0 eth2 10.128.0.0 169.254.20.109 255.255.0.0 UG 0 0 0 vti0 81.82.192.0 0.0.0.0 255.255.192.0 U 0 0 0 eth2 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo 169.254.20.88 0.0.0.0 255.255.255.252 U 0 0 0 vti1 169.254.20.108 0.0.0.0 255.255.255.252 U 0 0 0 vti0

Any idea how to correct this?

3

I think this is by design how iBGP works. When you admin shutdown the .89 peer does the route from .109 come into the table? Look at the BGP config for “maximum-paths” if you want to increase the number of parallel equal-cost paths allowed in the table.

set protocols bgp 60000 maximum-paths ebgp 2

4

@jl3128, how do you “admin shutdown the .89 peer”? Remove the peer from the config tree? Nevertheless, I tried your config setting but it didn’t change much in behavior. Still no traffic flowing from my network to the AWS VPC. :frowning:

5

in config mode you can do “set protocols bgp 60000 neighbor 169.254.20.89 shutdown” to disable the peering with the .89 peer

what about the exec mode output of “show ip bgp neighbors 169.254.20.109”

6

@jl3128,

Meanwhile, I have recreated the AWS VPN connection, so some of the IP address have changed. Here is the output for the command for both peers:

[code]vyos@vyos:~$ show ip bgp neighbors 169.254.21.137
BGP neighbor is 169.254.21.137, remote AS 9059, local AS 65000, external link
BGP version 4, remote router ID 169.254.21.137
BGP state = Established, up for 00:00:32
Last read 05:42:23, hold time is 30, keepalive interval is 10 seconds
Configured hold time is 30, keepalive interval is 10 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 8 4
Notifications: 1 0
Updates: 11 8
Keepalives: 471 464
Route Refresh: 0 0
Capability: 0 0
Total: 491 476
Minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast
Inbound soft reconfiguration allowed
Community attribute sent to this neighbor(both)
1 accepted prefixes

Connections established 7; dropped 6
Last reset 00:24:14, due to Admin. shutdown
Local host: 169.254.21.138, Local port: 59762
Foreign host: 169.254.21.137, Foreign port: 179
Nexthop: 169.254.21.138
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Read thread: on Write thread: off
[/code]

And

[code]vyos@vyos:~$ show ip bgp neighbors 169.254.21.221
BGP neighbor is 169.254.21.221, remote AS 9059, local AS 65000, external link
BGP version 4, remote router ID 169.254.21.221
BGP state = Established, up for 00:26:16
Last read 05:42:28, hold time is 30, keepalive interval is 10 seconds
Configured hold time is 30, keepalive interval is 10 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 7 3
Notifications: 0 0
Updates: 9 8
Keepalives: 613 607
Route Refresh: 0 0
Capability: 0 0
Total: 629 618
Minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast
Inbound soft reconfiguration allowed
Community attribute sent to this neighbor(both)
1 accepted prefixes

Connections established 6; dropped 5
Last reset 08:31:00, due to
Local host: 169.254.21.222, Local port: 179
Foreign host: 169.254.21.221, Foreign port: 37554
Nexthop: 169.254.21.222
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Read thread: on Write thread: off
[/code]

7

are you advertising your own subnets to the bgp peers? you can add a network statement under the bgp configuration

set protocols bgp 60000 network 172.16.20.0/24
set protocols bgp 60000 network 172.16.30.0/24
set protocols bgp 60000 network 172.16.35.0/24
set protocols bgp 60000 network 172.16.36.0/24
set protocols bgp 60000 network 172.16.37.0/24
set protocols bgp 60000 network 172.16.40.0/24
set protocols bgp 60000 network 172.16.50.0/24

you could also aggregate if you don’t need granularity 172.16.0.0/16 and depending what’s on the other side

8

@jl3128,

I have tried both without and with setting the bgp network to my local subnet (/16). Didn’t make a difference.

Ringo

9

you need to set the bgp network to make sure it is advertised to peers
what routes are you receiving?

10

@jl3128,

Here are the received routes,

vyos@vyos:~$ show ip bgp neighbors 169.254.21.221 received-routes
BGP table version is 0, local router ID is 172.16.20.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.128.0.0/16    169.254.21.221         200             0 9059 i

Total number of prefixes 1
vyos@vyos:~$ show ip bgp neighbors 169.254.21.137 received-routes
BGP table version is 0, local router ID is 172.16.20.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.128.0.0/16    169.254.21.137         100             0 9059 i

Total number of prefixes 1

As mentioned earlier, I tried the setup two times: once without setting the bgp network advertisement, and once with setting it. No success.

Ringo

11

So far, I always tried with the /16 one without success. This morning, I tried for the first time with a /24 subnet and it worked!

My ip routes on my VyOS:

[code]vyos@vyos:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route

S>* 0.0.0.0/0 [210/0] via , eth2
B>* 10.128.0.0/16 [20/100] via 169.254.21.137, vti0, 01w3d06h
C>* 81.82.192.0/18 is directly connected, eth2
C>* 127.0.0.0/8 is directly connected, lo
C>* 169.254.21.136/30 is directly connected, vti0
C>* 169.254.21.220/30 is directly connected, vti1
C>* 172.16.20.0/24 is directly connected, eth4
C>* 172.16.40.0/24 is directly connected, eth7
[/code]

Given the routes above (only 2 of the 7 subnets are really configured ATM), do you have any idea why it doesn’t work by passing the /16 subnet?

Ringo[/code]

12

Sounds like the AWS peer is configured to accept specific prefixes, maybe using a route-map. Do you set which prefixes you will be advertising to AWS in the AWS configuration?