Cannot get NAT Hairpinning to work

Hello there! Novice VyOS user here.

I switched my router from Unifi USG to a VyOS box, while it is much more stable I am stuck with 1 issue: I cannot reach services in my network via my public IP, while being inside my network. Outside the network this works perfectly fine. After doing some Googling, I found that NAT Hairpinning is the solution to this problem.

Now I noticed that this is explained quite well in the VyOS docs, but as is usual with networks my situation differs from the docs. I’ve got a record, lets call it example.com, pointing to my public IPv4, outside my network the service is reachable, inside it is not.

My subnet topology is as follows:
192.168.1.0/24 - default network, eth1
192.168.2.0.24 - Servers, eth1.20
10.10.4.0/24 - VMs and Kubernetes

I am on the default network myself, the service is on Kubernetes. At first connecting to the service resulted in a connection refused, so the traffic never even reached the server. Now however I’m getting a time out on the response, indicating that the traffic reached the server but isn’t coming back (correctly?).

My SNAT rules:

rule 100 {
description LAN-NAT
outbound-interface eth0.300
source {
address 192.168.1.0/24
}
translation {
address masquerade
}
}
rule 101 {
description VM-NAT
outbound-interface eth0.300
source {
address 10.10.4.0/24
}
translation {
address masquerade
}
}
rule 102 {
description SERVER-NAT
outbound-interface eth0.300
source {
address 192.168.2.0/24
}
translation {
address masquerade
}
}
rule 1000 {
description “NAT HTTP(S) Inside”
destination {
address 10.10.4.0/24
}
outbound-interface eth1.40
protocol tcp_udp
source {
address 10.10.4.0/24
}
translation {
address masquerade
}
}
rule 1001 {
destination {
address 192.168.1.0/24
}
outbound-interface eth1
protocol tcp_udp
source {
address 192.168.2.0/24
}
translation {
address masquerade
}
}

My DNAT rules:

rule 110 {
description “Kubernetes Ingress”
destination {
port 443
}
inbound-interface eth0.300
protocol tcp
translation {
address 10.10.4.254
port 443
}
}
rule 1000 {
destination {
address [PUBLIC v4]
port 443,80
}
inbound-interface eth1.40
protocol tcp_udp
translation {
address 10.10.4.254
}
}
rule 1001 {
destination {
address [PUBLIC v4]
port 443,80
}
inbound-interface eth1
protocol tcp_udp
source {
address 192.168.1.0/24
}
translation {
address 10.10.4.254
}
}

Help getting this to work would be greatly appreciated!

Hi

there are something that i don’t understand well , you say the default network (where you receive traffic on internet ) is on the eth1 , but when there are different interfaces in your SNAT- masquerade (0.X /1.X). Could you share the vyos version? and another question about the services ,they work well without NAT (between server / vms)

Hey, thanks for your response!

My WAN is eth0.300. What I mean with default network is where all clients that arent explicitly in another VLAN, so in 192.168.1.0/24 on eth1.

The other subnets all live on different VLANs, but on the same interface. Vlan 20 and 40 respectively.

I’m on VyOS 1.3.0-rc5 on amd64.

Yep communication between the servers, and even vlans is fine. But as soon as a DNS record resolves to my public v4, it breaks down

Rules look OK at first sight. Play around with tcpdump on in/outgoing interfaces to see what happens, also conntrack might reveal used NAT translations

there are rules that it doesn’t clear but there that understand well the flow and traffic origin or destination, example it:

rule 1000 {
description “NAT HTTP(S) Inside”
destination {
address 10.10.4.0/24
}
outbound-interface eth1.40
protocol tcp_udp
source {
address 10.10.4.0/24
}
translation {
address masquerade
}
}
rule 1001 {
destination {
address 192.168.1.0/24
}
outbound-interface eth1
protocol tcp_udp
source {
address 192.168.2.0/24
}
translation {
address masquerade
}

but is a good idea check with tcpdump and the conntrack states when you do your test .

You do need those masquerade rules on LAN interface for return traffic.
Without them, https server return traffic will directly be sent to client on LAN. With source IP address of HTTPS server on LAN , not WAN IP
Whereas the client initiated the connection to Server WAN IP, and thus is not expecting return traffic from different source IP.
With masquerade, https server sends back return traffic to VyOS, which translates back source and dest IPs

For now I’ve opted for split-brain DNS to resolve the issue and get around hairpinning alltogether, not the most optimal but, at least it works :smiley: