Hello, I am fairly new to this stuff.
I am trying to connect two Amazon regions (10.1.0.0/16 and 10.2.0.0/16) to VyOS (10.100.0.5) .
BGP advertising is working fine:
Here is the route information:
I can ping 10.1.0.5/10.2.0.5 to/from 10.100.0.5 (VyOS)
But I am unable to ping 10.1.0.5 to/from 10.2.0.5
VyOS is connected to AWS regions over AWS VPC VPNs with dynamic routing.
I pretty much imported Vyatta configuration from AWS VPN. Attached is config.boot
Appreciate any help/pointers. Thanks
for i in 169.254.253.17 169.254.253.21 69.254.255.1 69.254.255.5; do
show ip bgp neighbors $i advertised-routes
show ip bgp neighbors $i received-routes
doneI think you are only advertising your own 10.100.0.0/25, and 10/8 for which you have a local blackhole route. 10.1.0.0/16 and 10.2.0.0/16 were presumably learned from one of the 4 peers, but 1) are you actually advertising that learned route back into the other peers, and 2) will AS7224 accept those routes from you and use your vyos box as transit to reach those networks? I am not familiar with how AWS does such internal routing.
Carl, thank you for your reply. AWS is getting the 10.0.0.0/8 route. But not the 10.x.0.0/16 routes. I see this in AWS VPC console:
If I remove the static network with blackhole, no networks are showing up on AWS side.
vyos@VyOS-AMI:~$ show ip bgp
BGP table version is 0, local router ID is 10.100.0.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 10.0.0.0 0.0.0.0 0 32768 i
* 10.1.0.0/16 169.254.253.17 0 7224 i
*> 169.254.253.21 0 7224 i
* 10.2.0.0/16 169.254.255.1 0 7224 i
*> 169.254.255.5 0 7224 i
Total number of prefixes 3vyos@VyOS-AMI:~$ for i in 169.254.253.17 169.254.253.21 169.254.255.1 169.254.255.5; do
echo "Advertised to: $i"
show ip bgp neighbors $i advertised-routes
echo "Received from: $i"
show ip bgp neighbors $i received-routes
done
Advertised to: 169.254.253.17
*> 10.0.0.0 169.254.253.18 0 32768 i
*> 10.1.0.0/16 169.254.253.18 0 7224 i
*> 10.2.0.0/16 169.254.253.18 0 7224 i
Received from: 169.254.253.17
*> 10.1.0.0/16 169.254.253.17 0 7224 i
Advertised to: 169.254.253.21
*> 10.0.0.0 169.254.253.22 0 32768 i
*> 10.2.0.0/16 169.254.253.22 0 7224 i
Received from: 169.254.253.21
*> 10.1.0.0/16 169.254.253.21 0 7224 i
Advertised to: 169.254.255.1
*> 10.0.0.0 169.254.255.2 0 32768 i
*> 10.1.0.0/16 169.254.255.2 0 7224 i
*> 10.2.0.0/16 169.254.255.2 0 7224 i
Received from: 169.254.255.1
*> 10.2.0.0/16 169.254.255.1 0 7224 i
Advertised to: 169.254.255.5
*> 10.0.0.0 169.254.255.6 0 32768 i
*> 10.1.0.0/16 169.254.255.6 0 7224 i
Received from: 169.254.255.5
*> 10.2.0.0/16 169.254.255.5 0 7224 iSorry, I had to truncate the output. With full output from “for loop”, forum was flagging it as spam … 
Anyone have clues? According to Google, looks like someone using Cisco ASA hit similar issue. And the solution was to use “same-security-traffic permit intra-interface”. Is there a similar setting in VyOS? Thanks
I don’t think that is the problem. ASA devices have many strange rules - vyos is more of a pure routing platform. You are advertising the right routes back into AWS, but they
may be filtering your advertisements.
From 10.1.0.5, what does ‘traceroute 10.2.0.5’ show? Does it go
thru any of the 169.254.255.x bgp peers?
From 10.1.0.5, what does ‘traceroute 10.100.0.5’ show? That should end
up at your vyos system.
From 10.1.0.5, run ‘ping 10.2.0.5’, and at the same time, on vyos:
sudo tcpdump -i vti0 host 10.1.0.5
sudo tcpdump -i vti1 host 10.1.0.5The pings from 10.1.0.5 should arrive via either vti0 or vti1. If those pings don’t arrive at the vyos system, then I think that AWS is filtering your route announcements.
Carl, thanks again for your response. Here is the output for the commands. I removed all traceroute output which are just “* * *”
[root@ip-10-1-0-5 ~]# traceroute 10.2.0.5
traceroute to 10.2.0.5 (10.2.0.5), 30 hops max, 60 byte packets
1 169.254.253.9 (169.254.253.9) 0.329 ms 0.566 ms 169.254.253.13 (169.254.253.13) 0.364 ms
2 169.254.253.21 (169.254.253.21) 0.582 ms 0.824 ms 0.566 ms
3 10.100.0.5 (10.100.0.5) 2.706 ms 2.684 ms 2.910 ms
5 10.100.0.5 (10.100.0.5) 3.405 ms 3.636 ms 3.391 ms
11 10.100.0.5 (10.100.0.5) 5.582 ms 5.564 ms 5.561 ms
13 * 10.100.0.5 (10.100.0.5) 6.944 ms 6.812 ms
17 * 10.100.0.5 (10.100.0.5) 8.097 ms 8.312 ms
19 10.100.0.5 (10.100.0.5) 9.065 ms 9.052 ms 8.803 ms
25 10.100.0.5 (10.100.0.5) 12.697 ms 12.936 ms 12.473 ms
27 10.100.0.5 (10.100.0.5) 13.398 ms 13.380 ms *
28 29 30[root@ip-10-2-0-5 ~]# traceroute 10.100.0.5
traceroute to 10.100.0.5 (10.100.0.5), 30 hops max, 60 byte packets
1 169.254.255.33 (169.254.255.33) 2.445 ms 169.254.255.29 (169.254.255.29) 1.689 ms 169.254.255.33 (169.254.255.33) 2.407 ms
2 169.254.255.5 (169.254.255.5) 1.662 ms 1.675 ms 1.647 ms
3 10.100.0.5 (10.100.0.5) 82.866 ms 82.695 ms 82.710 msWhen I ping from 10.1.0.5 to 10.2.0.5, tcpdump traffic on vti0 or vti1 is empty. But there is traffic on eth0.
[root@ip-10-1-0-5 ~]# ping 10.2.0.5
PING 10.2.0.5 (10.2.0.5) 56(84) bytes of data.
^C
--- 10.2.0.5 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1660msroot@VyOS-AMI:~# sudo tcpdump -i vti0 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti0, link-type RAW (Raw IP), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
root@VyOS-AMI:~# sudo tcpdump -i vti1 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti1, link-type RAW (Raw IP), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
root@VyOS-AMI:~# sudo tcpdump -i eth0 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
21:56:35.903177 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
21:56:35.903894 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
21:56:35.904656 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
21:56:35.905433 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
.....But, if I ping from 10.1.0.5 to 10.100.0.5, I see ICMP echo reply traffic on vti0
root@VyOS-AMI:~# sudo tcpdump -i vti0 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti0, link-type RAW (Raw IP), capture size 65535 bytes
21:57:51.299033 IP 10.100.0.5 > 10.1.0.5: ICMP echo reply, id 24861, seq 1, length 64
21:57:52.300578 IP 10.100.0.5 > 10.1.0.5: ICMP echo reply, id 24861, seq 2, length 64
.....
^CThat looks like some sort of routing loop. You have two links into each of 10.x.0.0/16. Can you take down one on each side - and does that improve things? If so, try adding route-maps to block the 10.1.0.0/16 advertised-route to vti0/1 and block the 10.2.0.0/16 advertised-route to vti2/3.
Carl, thanks for your response.
I disabled vti1 and vit3 as you suggested. And things are a bit different now. I still can’t ping all the way through. But traceroute show one extra hop. The 204.246.160.78 address below is the public IP address of vti2.
It looks like traffic is coming in from vti0 and leaving vti2. For some reason, I still cannot see any of the traffic on vti using tcpdump. I only see tcpdump output on eth0. I will play with route-map as you suggested.
traceroute to 10.2.0.5 (10.2.0.5), 30 hops max, 60 byte packets
1 169.254.253.13 (169.254.253.13) 0.557 ms 169.254.253.9 (169.254.253.9) 0.471 ms 169.254.253.13 (169.254.253.13) 2.557 ms
2 169.254.253.17 (169.254.253.17) 2.303 ms 2.050 ms 2.069 ms
3 10.100.0.5 (10.100.0.5) 4.554 ms 4.280 ms 4.516 ms
4 204.246.160.78 (204.246.160.78) 4.348 ms 4.336 ms 4.543 ms
5 10.100.0.5 (10.100.0.5) 6.545 ms 6.520 ms 6.770 ms
6 204.246.160.78 (204.246.160.78) 6.841 ms 6.448 ms *
7 * * *
8 204.246.160.78 (204.246.160.78) 8.176 ms 8.227 ms 7.878 ms
9 * * *
10 204.246.160.78 (204.246.160.78) 10.469 ms * 9.930 ms
11 * * *
12 204.246.160.78 (204.246.160.78) 12.337 ms * 12.300 ms
13 * * *
14 * 204.246.160.78 (204.246.160.78) 14.721 ms *
15 10.100.0.5 (10.100.0.5) 17.070 ms 16.884 ms 16.894 ms
16 204.246.160.78 (204.246.160.78) 17.426 ms 17.401 ms 16.860 ms
17 10.100.0.5 (10.100.0.5) 19.616 ms 19.087 ms *
18 204.246.160.78 (204.246.160.78) 19.873 ms * *
19 * * *
20 204.246.160.78 (204.246.160.78) 20.881 ms 20.641 ms 20.441 ms
21 * * *
22 204.246.160.78 (204.246.160.78) 22.741 ms 22.474 ms *
23 * * *
24 204.246.160.78 (204.246.160.78) 24.892 ms 25.611 ms *
25 * 10.100.0.5 (10.100.0.5) 27.212 ms 27.208 ms
26 204.246.160.78 (204.246.160.78) 28.109 ms 27.254 ms 27.179 ms
27 10.100.0.5 (10.100.0.5) 29.282 ms 29.762 ms 29.772 ms
28 204.246.160.78 (204.246.160.78) 29.326 ms 29.536 ms *
29 * * *
30 204.246.160.78 (204.246.160.78) 31.803 ms 31.781 ms 29.831 msHello. Is your problem this vti’s bug?
Bug 358 - Can’t reach other side of VTI IPsec tunnel but can see packets on VTI interface
http://bugzilla.vyos.net/show_bug.cgi?id=358
This bug occurs vyos v1.1.*.
shunsuke, it looks like it. Thanks.
I was under the impression it was fixed in 1.1.1. I was using 1.1.1. Apparently, it is still broken. Downgraded to 1.0.5 and everything works as expected.
Carl, thanks for your time.