Openstack & Vyos

General questions
,
1

Hi all…

I’m trying to get VyOS to run in an Openstack instance (GRE With VLANS)…

I have the interfaces up & running, all is good there.

VYOS router has one directly attached interface to an external network, and one interface attached to the internal interace.

Internet works on the external interface without issues.

  • I can ping outside machines (8.8.8.8)

Network works on internal interface without issues.

  • I can ping internal address no problem (10.168.11.102)

But if I try and route through the VyOS, something doesn’t make it through.

I can see the packets through tcpdump on both interfaces, but the destination machine never sees the inbound packets! (10.168.11.102)

vyos@vyos:~$ /usr/sbin/tcpdump -f “icmp” -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:10:13.234909 IP 10.168.11.102 > 8.8.8.8: ICMP echo request, id 40962, seq 49, length 64
18:10:13.261277 IP 8.8.8.8 > 10.168.11.102: ICMP echo reply, id 40962, seq 49, length 64
18:10:14.235045 IP 10.168.11.102 > 8.8.8.8: ICMP echo request, id 40962, seq 50, length 64
18:10:14.261379 IP 8.8.8.8 > 10.168.11.102: ICMP echo reply, id 40962, seq 50, length 64
18:10:15.235249 IP 10.168.11.102 > 8.8.8.8: ICMP echo request, id 40962, seq 51, length 64
18:10:15.261549 IP 8.8.8.8 > 10.168.11.102: ICMP echo reply, id 40962, seq 51, length 64
^C
6 packets captured
6 packets received by filter
0 packets dropped by kernel
vyos@vyos:~$ /usr/sbin/tcpdump -f “icmp” -i eth1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
18:10:18.235887 IP XXX.XXX.187.78 > 8.8.8.8: ICMP echo request, id 40962, seq 54, length 64
18:10:18.262249 IP 8.8.8.8 > XXX.XXX.187.78: ICMP echo reply, id 40962, seq 54, length 64
18:10:19.236110 IP XXX.XXX.187.78 > 8.8.8.8: ICMP echo request, id 40962, seq 55, length 64
18:10:19.262477 IP 8.8.8.8 > XXX.XXX.187.78: ICMP echo reply, id 40962, seq 55, length 64
18:10:20.236345 IP XXX.XXX.187.78 > 8.8.8.8: ICMP echo request, id 40962, seq 56, length 64
18:10:20.262652 IP 8.8.8.8 > XXX.XXX.187.78: ICMP echo reply, id 40962, seq 56, length 64
18:10:21.236527 IP XXX.XXX.187.78 > 8.8.8.8: ICMP echo request, id 40962, seq 57, length 64
18:10:21.262927 IP 8.8.8.8 > XXX.XXX.187.78: ICMP echo reply, id 40962, seq 57, length 64
18:10:22.237082 IP XXX.XXX.187.78 > 8.8.8.8: ICMP echo request, id 40962, seq 58, length 64
18:10:22.263398 IP 8.8.8.8 > XXX.XXX.187.78: ICMP echo reply, id 40962, seq 58, length 64
^C
10 packets captured
10 packets received by filter
0 packets dropped by kernel
vyos@vyos:~$ ping 10.168.11.102
PING 10.168.11.102 (10.168.11.102) 56(84) bytes of data.
64 bytes from 10.168.11.102: icmp_req=1 ttl=64 time=0.481 ms
64 bytes from 10.168.11.102: icmp_req=2 ttl=64 time=0.559 ms

Any Ideas? 10.168.11.102 never sees the responses…

2

You must setup a static ip on your vyos lan interface . If eth0 is the lan interface then your setup is wrong

interfaces {
ethernet eth0 {
address dhcp
duplex auto
smp_affinity auto
speed auto
}

3

I’ll give that a shot, thanks!

Unfortunately, It didn’t work…

I did notice something…

Incoming packets have a tos of 0x28 and ttl 46… (second packet is response from outside)
vyos@vyos:~$ /usr/sbin/tcpdump -f “icmp” -i eth1 -vv

tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
14:34:40.239401 IP (tos 0x0, ttl 63, id 29883, offset 0, flags [DF], proto ICMP (1), length 84)
38.64.187.78 > 8.8.8.8: ICMP echo request, id 16908, seq 0, length 64
14:34:40.267924 IP (tos 0x28, ttl 46, id 0, offset 0, flags [none], proto ICMP (1), length 84)
8.8.8.8 > 38.64.187.78: ICMP echo reply, id 16908, seq 0, length 64

where as the packets that work, have tos of 0x00 and ttl 64
vyos@vyos:~$ /usr/sbin/tcpdump -f “icmp” -i eth0 -vv
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
14:39:49.285985 IP (tos 0x0, ttl 64, id 29887, offset 0, flags [DF], proto ICMP (1), length 84)
10.168.11.102 > 8.8.8.8: ICMP echo request, id 20492, seq 0, length 64
14:39:49.314824 IP (tos 0x28, ttl 45, id 0, offset 0, flags [none], proto ICMP (1), length 84)
8.8.8.8 > 10.168.11.102: ICMP echo reply, id 20492, seq 0, length 64

Would that make a difference?

4

FYI… found how to change TTL & TOS set to 64 & 0x0 respectively…

still not working… (for some reason… the TTL when set to 64 was 63 in packets… set to 65 in table, and came out to 64 in packets)

vyos@vyos:~$ sudo iptables -t mangle -L -n -x -v
Chain PREROUTING (policy ACCEPT 15 packets, 1012 bytes)
pkts bytes target prot opt in out source destination
650 43832 DSCP all – * * 0.0.0.0/0 0.0.0.0/0 DSCP set 0x00
82 5728 TTL all – * * 0.0.0.0/0 0.0.0.0/0 TTL set to 65

Chain INPUT (policy ACCEPT 15 packets, 1012 bytes)
pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 10 packets, 1056 bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 10 packets, 1056 bytes)
pkts bytes target prot opt in out source destination
vyos@vyos:~$ /usr/sbin/tcpdump -f “icmp” -i eth0 -vv
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:05:15.906793 IP (tos 0x0, ttl 64, id 29920, offset 0, flags [DF], proto ICMP (1), length 84)
10.168.11.102 > 8.8.8.8: ICMP echo request, id 35340, seq 0, length 64
15:05:15.935631 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto ICMP (1), length 84)
8.8.8.8 > 10.168.11.102: ICMP echo reply, id 35340, seq 0, length 64

5

openstack enable spoofing filter setting

http://docs.openstack.org/api/openstack-network/2.0/content/allowed_address_pair_ext.html

probably source ip filterd.

///example

#neutron port-update {port-id} --allowed-address-pairs type=dict list=true ip_address=0.0.0.0/0

#neutron port-show {port-id}

6

Hi,
I have the same problem too,and I wonder if you have solved this problem,If you fixed it,please let me know.Thank you very much

7

Well, my problem is resolved.

8

Could you tell me how you solve this problem,please?

9

In Openstack, you have unicast RPF mechanism to avoid communication from an unknown source IP.
On your openstack, you have to play with the “allowed address pairs” to allow a flow from a source not the IP of the interface of the VM itself.
First, check you have this extension with the command neutron ext-list and verify you have “allowed address pairs” listed.
2nd, update the port of your VM with a command like this one.
neutron port-update ‘port-id’ --allowed_address_pairs list=true type=dict ip_address=0.0.0.0/0 (you can restrict to a specific subnet)

Hope it helps.

10