Ping Replies only when tcp monitor is on?

Bugs
, , , , ,
1

Hi Guys,
I am experiencing the most bizzare issue in the history of my networking and I hope you can help.
I have the UBNT Router and VyOS router.
No Firewall filters at both ends.
I can post the config of both if necessary… but hear me first :slight_smile:

The UBNT is trying to ping the Public IP address of VyOS - Nothing.
VyOS is trying to ping the public IP address of UBNT - nothing

I go to VYOS and type:

monitor interfaces ethernet eth0 traffic filter “host PUBLIC IP of UBNT ROUTER”

VyOS CLI:
Taking a bit of time… and then:

Capturing traffic on eth0 …
0.000000 176.118.139.48 → 185.137.20.99 ICMP Echo (ping) request
0.000033 185.137.20.99 → 176.118.139.48 ICMP Echo (ping) reply
While the capture is running like this… you go to UBNT router CLI and you can see the reply !
You stop the monitor, the PING immediately stops.

But wait it gets better.

When PING replies, the GRE tunnel between them comes up and it’s all working :slight_smile:

Then , I go to VYOS and I stop the capture - all drops, no GRE tunnel, no PING.
I can keep the network “alive” only when I monitor ETH0 interface.
Its just CRAZY.
Please help.

2

Linux vyos 3.13.11-1-amd64-vyos #1 SMP Sat Nov 11 12:10:30 CET 2017 x86_64

vyos@vyos:~$ show version
Version: VyOS 1.1.8
Description: VyOS 1.1.8 (helium)
Copyright: 2017 VyOS maintainers and contributors
Built by: maintainers@vyos.net
Built on: Sat Nov 11 13:44:36 UTC 2017
Build ID: 1711111344-b483efc
System type: x86 64-bit
Boot via: image
Hypervisor: VMware
HW model: VMware Virtual Platform
HW S/N: VMware-56 4d ae f6 8d fd ba 64-06 76 dd b1 5d a5 39 3c
HW UUID: 564DAEF6-8DFD-BA64-0676-DDB15DA5393C
Uptime: 18:57:01 up 22 days, 1:56, 2 users, load average: 0.02, 0.08, 0.07

VYOS BASIC CONFIG

Incomplete command: show

vyos@vyos:~$ configure
s[edit]
vyos@vyos# show
firewall {
all-ping enable
}
interfaces {
ethernet eth0 {
address INTERNET WAN IP
description INTERNET
duplex auto
hw-id 00:0c:29:a5:39:3c
smp_affinity auto
speed auto
}
ethernet eth1 {
address 169.254.10.1/30
description IPSON-VM100
duplex auto
hw-id 00:0c:29:a5:39:46
smp_affinity auto
speed auto
}
ethernet eth2 {
address 10.250.252.25/24
description MGMT
duplex auto
hw-id 00:0c:29:a5:39:50
smp_affinity auto
speed auto
}
loopback lo {
}
tunnel tun0 {
address 169.254.1.1/30
encapsulation gre
local-ip 185.137.20.99
multicast disable
remote-ip 193.189.67.54
}
tunnel tun1 {
address 169.254.2.1/30
encapsulation gre
local-ip 185.137.20.99
multicast disable
remote-ip 86.40.142.105
}
tunnel tun2 {
address 169.254.3.1/30
encapsulation gre
local-ip 185.137.20.99
multicast disable
remote-ip 176.118.139.48
}
}
protocols {
static {
route 0.0.0.0/0 {
next-hop 169.254.10.2 {
}
}
route 10.101.10.0/24 {
next-hop 10.250.252.1 {
}
}
route 86.40.142.105/32 {
next-hop 185.137.20.1 {
}
}
route UBNT ROUTER WAN IP {
next-hop 185.137.20.1 {
}
}
route 192.168.210.0/24 {
next-hop 169.254.2.2 {
}
}
route 192.168.234.0/24 {
next-hop 169.254.3.2 {
}
}
route 192.168.240.0/24 {
next-hop 169.254.1.2 {
}
}
route 193.189.67.52/30 {
next-hop DEFAULT WAN GATEWAY {
}
}
route 193.189.67.54/32 {
next-hop DEFAULT WAN GATEWAY {
}
}
}
}
service {
dns {
}
ssh {
port 22
}
}
system {
config-management {
commit-revisions 20
}
console {
device ttyS0 {
speed 9600
}
}
host-name vyos
login {
user vyos {
authentication {
encrypted-password $6$sl/NUej24mJ$Sn1d1qMKSqp.iZ03dPKfTvDsmadTRyyWP0hZpfysoLqoFNMqK9rQYNC/AKzSOa0r80IJMKULORv/4Qvj53LME1
plaintext-password “”
}
level admin
}
}
name-server 8.8.8.8
ntp {
server 0.pool.ntp.org {
}
server 1.pool.ntp.org {
}
server 2.pool.ntp.org {
}
}
package {
auto-sync 1
repository community {
components main
distribution helium
password “”
url http://packages.vyos.net/vyos
username “”
}
}
syslog {
global {
facility all {
level notice
}
facility protocols {
level debug
}
}
}
time-zone UTC
}
[edit]
vyos@vyos#

3

Hey,
i will suggest you to use 1.2 either rolling or rc5
as we not really fixing anything in 1.1.x

4

I have just already upgraded to 1.2
No difference.
It’s very strange.

Maybe it’s something with the setup because I’m using 169.254 address space for GRE tunneling
But why would everything work while I have the monitor on - I have no idea.
The minute I press CTRL+C to terminate the monitor , eveyrhing stops.

Is there something I can capture ( tech support files, conifg files) to escalate this issue to engineering or I have no hope?

I’m asking because we are building cloud solution on Edge Routers from UBNT and we were hoping to have VyOS GRE terminator in front of our firewall in the Vmware stack.
But the most recent discovery is puzzling us… especially with the fact that it worked with 2 other tunnels perfectly.

VyOS has 3 GRE tunnels now, the first two worked perfectly and we just put 3rd and we started to see this issue.

is there any info I can provide that could help ?

5

Hello, @MP_IRL!
Before examination any other nuances of this issue you definitely must change all 169.254.0.0/16 addresses to some other network.
According to RFC3927 there are a lot of limitations to using addresses from this network.

6

Hi There,
So I have changed the subnets, but the issue remains.
Please note, that we can actually forget about these GRE tunnels completely, simple pings don’t work.

Here are the IPs:
Public IP of VYOS: 185.137.20.99
Public IP of UBNT Edge Router: 176.118.139.48

I try to ping the Public IP from either direction ( no firewall ) - Fail.
Now I do magic trick !

I type on VYOS:
monitor traffic interface eth0 filter “host 1.1.1.1”
I picked 1.1.1.1 so I have no hits, it doesn’t matter that I don’t have any hits, what matters is that the VyOS is monitoring this interface.

The PING between the locations work perfectly, the GRE tunnel comes up, all working perfectly.
It’s actually working so my priority decreased but imagine how silly this is.

If I personally would have worked in the VyOS development team, I would be very curious what is happening…

7

Hi, @MP_IRL!
Can you try for testing purposes execute next command in shell instead of tcpdump?
sudo ip link set eth0 promisc on
Check connectivity after this please.

8

That worked also !

Removed my monitor.
Entered the command you’ve provided.
All up and running.

But why would it have worked for 3 public IP addresses and not for 1 - mystery.

I can confirm that at this moment everything is working well without monitor on.

9

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

10

Hello, @MP_IRL!
This is not looks like bug. It more seems that there it is some misconfiguration or incompatibility between VyOS Ethernet interface settings and hypervisor or router behind this interface. Like traffic for VyOS is sending with wrong MAC address.
Check all settings and maybe you find what can cause this.