Forwarding traffic


#1

Hello, I am fairly new to this stuff.

I am trying to connect two Amazon regions (10.1.0.0/16 and 10.2.0.0/16) to VyOS (10.100.0.5) .

BGP advertising is working fine:

Here is the route information:

I can ping 10.1.0.5/10.2.0.5 to/from 10.100.0.5 (VyOS)

But I am unable to ping 10.1.0.5 to/from 10.2.0.5

VyOS is connected to AWS regions over AWS VPC VPNs with dynamic routing.

I pretty much imported Vyatta configuration from AWS VPN. Attached is config.boot

Appreciate any help/pointers. Thanks


#2
for i in 169.254.253.17 169.254.253.21 69.254.255.1 69.254.255.5; do 
    show ip bgp neighbors $i advertised-routes
    show ip bgp neighbors $i received-routes
done

I think you are only advertising your own 10.100.0.0/25, and 10/8 for which you have a local blackhole route. 10.1.0.0/16 and 10.2.0.0/16 were presumably learned from one of the 4 peers, but 1) are you actually advertising that learned route back into the other peers, and 2) will AS7224 accept those routes from you and use your vyos box as transit to reach those networks? I am not familiar with how AWS does such internal routing.


#3

Carl, thank you for your reply. AWS is getting the 10.0.0.0/8 route. But not the 10.x.0.0/16 routes. I see this in AWS VPC console:

If I remove the static network with blackhole, no networks are showing up on AWS side.

vyos@VyOS-AMI:~$ show ip bgp
BGP table version is 0, local router ID is 10.100.0.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.0.0         0.0.0.0                  0         32768 i
*  10.1.0.0/16      169.254.253.17                         0 7224 i
*>                  169.254.253.21                         0 7224 i
*  10.2.0.0/16      169.254.255.1                          0 7224 i
*>                  169.254.255.5                          0 7224 i

Total number of prefixes 3
vyos@VyOS-AMI:~$ for i in 169.254.253.17 169.254.253.21 169.254.255.1 169.254.255.5; do
    echo "Advertised to: $i"
    show ip bgp neighbors $i advertised-routes
    echo "Received from: $i"
    show ip bgp neighbors $i received-routes
done

Advertised to: 169.254.253.17
*> 10.0.0.0         169.254.253.18           0         32768 i
*> 10.1.0.0/16      169.254.253.18                         0 7224 i
*> 10.2.0.0/16      169.254.253.18                         0 7224 i

Received from: 169.254.253.17
*> 10.1.0.0/16      169.254.253.17                         0 7224 i

Advertised to: 169.254.253.21
*> 10.0.0.0         169.254.253.22           0         32768 i
*> 10.2.0.0/16      169.254.253.22                         0 7224 i

Received from: 169.254.253.21
*> 10.1.0.0/16      169.254.253.21                         0 7224 i

Advertised to: 169.254.255.1
*> 10.0.0.0         169.254.255.2            0         32768 i
*> 10.1.0.0/16      169.254.255.2                          0 7224 i
*> 10.2.0.0/16      169.254.255.2                          0 7224 i

Received from: 169.254.255.1
*> 10.2.0.0/16      169.254.255.1                          0 7224 i

Advertised to: 169.254.255.5
*> 10.0.0.0         169.254.255.6            0         32768 i
*> 10.1.0.0/16      169.254.255.6                          0 7224 i

Received from: 169.254.255.5
*> 10.2.0.0/16      169.254.255.5                          0 7224 i

Sorry, I had to truncate the output. With full output from “for loop”, forum was flagging it as spam … :frowning:


#4

Anyone have clues? According to Google, looks like someone using Cisco ASA hit similar issue. And the solution was to use “same-security-traffic permit intra-interface”. Is there a similar setting in VyOS? Thanks


#5

I don’t think that is the problem. ASA devices have many strange rules - vyos is more of a pure routing platform. You are advertising the right routes back into AWS, but they
may be filtering your advertisements.

From 10.1.0.5, what does ‘traceroute 10.2.0.5’ show? Does it go
thru any of the 169.254.255.x bgp peers?

From 10.1.0.5, what does ‘traceroute 10.100.0.5’ show? That should end
up at your vyos system.

From 10.1.0.5, run ‘ping 10.2.0.5’, and at the same time, on vyos:

sudo tcpdump -i vti0 host 10.1.0.5
sudo tcpdump -i vti1 host 10.1.0.5

The pings from 10.1.0.5 should arrive via either vti0 or vti1. If those pings don’t arrive at the vyos system, then I think that AWS is filtering your route announcements.


#6

Carl, thanks again for your response. Here is the output for the commands. I removed all traceroute output which are just “* * *”

[root@ip-10-1-0-5 ~]# traceroute 10.2.0.5
traceroute to 10.2.0.5 (10.2.0.5), 30 hops max, 60 byte packets
 1  169.254.253.9 (169.254.253.9)  0.329 ms  0.566 ms 169.254.253.13 (169.254.253.13)  0.364 ms
 2  169.254.253.21 (169.254.253.21)  0.582 ms  0.824 ms  0.566 ms
 3  10.100.0.5 (10.100.0.5)  2.706 ms  2.684 ms  2.910 ms
 5  10.100.0.5 (10.100.0.5)  3.405 ms  3.636 ms  3.391 ms
11  10.100.0.5 (10.100.0.5)  5.582 ms  5.564 ms  5.561 ms
13  * 10.100.0.5 (10.100.0.5)  6.944 ms  6.812 ms
17  * 10.100.0.5 (10.100.0.5)  8.097 ms  8.312 ms
19  10.100.0.5 (10.100.0.5)  9.065 ms  9.052 ms  8.803 ms
25  10.100.0.5 (10.100.0.5)  12.697 ms  12.936 ms  12.473 ms
27  10.100.0.5 (10.100.0.5)  13.398 ms  13.380 ms *
28 29 30
[root@ip-10-2-0-5 ~]# traceroute 10.100.0.5
traceroute to 10.100.0.5 (10.100.0.5), 30 hops max, 60 byte packets
 1  169.254.255.33 (169.254.255.33)  2.445 ms 169.254.255.29 (169.254.255.29)  1.689 ms 169.254.255.33 (169.254.255.33)  2.407 ms
 2  169.254.255.5 (169.254.255.5)  1.662 ms  1.675 ms  1.647 ms
 3  10.100.0.5 (10.100.0.5)  82.866 ms  82.695 ms  82.710 ms

When I ping from 10.1.0.5 to 10.2.0.5, tcpdump traffic on vti0 or vti1 is empty. But there is traffic on eth0.

[root@ip-10-1-0-5 ~]# ping 10.2.0.5
PING 10.2.0.5 (10.2.0.5) 56(84) bytes of data.
^C
--- 10.2.0.5 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1660ms
root@VyOS-AMI:~# sudo tcpdump -i vti0 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti0, link-type RAW (Raw IP), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

root@VyOS-AMI:~# sudo tcpdump -i vti1 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti1, link-type RAW (Raw IP), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

root@VyOS-AMI:~# sudo tcpdump -i eth0 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
21:56:35.903177 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
21:56:35.903894 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
21:56:35.904656 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
21:56:35.905433 IP 10.1.0.5 > 10.2.0.5: ICMP echo request, id 24605, seq 1, length 64
.....

But, if I ping from 10.1.0.5 to 10.100.0.5, I see ICMP echo reply traffic on vti0

root@VyOS-AMI:~# sudo tcpdump -i vti0 host 10.1.0.5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti0, link-type RAW (Raw IP), capture size 65535 bytes
21:57:51.299033 IP 10.100.0.5 > 10.1.0.5: ICMP echo reply, id 24861, seq 1, length 64
21:57:52.300578 IP 10.100.0.5 > 10.1.0.5: ICMP echo reply, id 24861, seq 2, length 64
.....
^C

#7

That looks like some sort of routing loop. You have two links into each of 10.x.0.0/16. Can you take down one on each side - and does that improve things? If so, try adding route-maps to block the 10.1.0.0/16 advertised-route to vti0/1 and block the 10.2.0.0/16 advertised-route to vti2/3.


#8

Carl, thanks for your response.

I disabled vti1 and vit3 as you suggested. And things are a bit different now. I still can’t ping all the way through. But traceroute show one extra hop. The 204.246.160.78 address below is the public IP address of vti2.

It looks like traffic is coming in from vti0 and leaving vti2. For some reason, I still cannot see any of the traffic on vti using tcpdump. I only see tcpdump output on eth0. I will play with route-map as you suggested.

traceroute to 10.2.0.5 (10.2.0.5), 30 hops max, 60 byte packets
 1  169.254.253.13 (169.254.253.13)  0.557 ms 169.254.253.9 (169.254.253.9)  0.471 ms 169.254.253.13 (169.254.253.13)  2.557 ms
 2  169.254.253.17 (169.254.253.17)  2.303 ms  2.050 ms  2.069 ms
 3  10.100.0.5 (10.100.0.5)  4.554 ms  4.280 ms  4.516 ms
 4  204.246.160.78 (204.246.160.78)  4.348 ms  4.336 ms  4.543 ms
 5  10.100.0.5 (10.100.0.5)  6.545 ms  6.520 ms  6.770 ms
 6  204.246.160.78 (204.246.160.78)  6.841 ms  6.448 ms *
 7  * * *
 8  204.246.160.78 (204.246.160.78)  8.176 ms  8.227 ms  7.878 ms
 9  * * *
10  204.246.160.78 (204.246.160.78)  10.469 ms *  9.930 ms
11  * * *
12  204.246.160.78 (204.246.160.78)  12.337 ms *  12.300 ms
13  * * *
14  * 204.246.160.78 (204.246.160.78)  14.721 ms *
15  10.100.0.5 (10.100.0.5)  17.070 ms  16.884 ms  16.894 ms
16  204.246.160.78 (204.246.160.78)  17.426 ms  17.401 ms  16.860 ms
17  10.100.0.5 (10.100.0.5)  19.616 ms  19.087 ms *
18  204.246.160.78 (204.246.160.78)  19.873 ms * *
19  * * *
20  204.246.160.78 (204.246.160.78)  20.881 ms  20.641 ms  20.441 ms
21  * * *
22  204.246.160.78 (204.246.160.78)  22.741 ms  22.474 ms *
23  * * *
24  204.246.160.78 (204.246.160.78)  24.892 ms  25.611 ms *
25  * 10.100.0.5 (10.100.0.5)  27.212 ms  27.208 ms
26  204.246.160.78 (204.246.160.78)  28.109 ms  27.254 ms  27.179 ms
27  10.100.0.5 (10.100.0.5)  29.282 ms  29.762 ms  29.772 ms
28  204.246.160.78 (204.246.160.78)  29.326 ms  29.536 ms *
29  * * *
30  204.246.160.78 (204.246.160.78)  31.803 ms  31.781 ms  29.831 ms

#9

Hello. Is your problem this vti’s bug?

Bug 358 - Can’t reach other side of VTI IPsec tunnel but can see packets on VTI interface
http://bugzilla.vyos.net/show_bug.cgi?id=358

This bug occurs vyos v1.1.*.


#10

shunsuke, it looks like it. Thanks.

I was under the impression it was fixed in 1.1.1. I was using 1.1.1. Apparently, it is still broken. Downgraded to 1.0.5 and everything works as expected.

Carl, thanks for your time.