IPSEC SNAT problem

200.X.X.X —> 187.X.X.X (peer) <-----------ipsec------------> 185.X.X.X (VyOS,peer) --NAT–> 192.168.X.21

Actors:
Left side IP 200.X.X.X, sends telnet probes each minute to 194.X.X.3:443 (DNAT/SNAT) which shoudl reach 192.168.X.21
The left site is configured by partner and I have no access to it.

Right side:
VyOS 1.3 eth0 (185.X.X.X), eth2 (192.168.X.1), dummy interface (194.X.X.3)
APP server eth1 (192.168.X.21)

The aim is to follow the way configuration is done https://docs.vyos.io/en/latest/nat.html “NAT before VPN”

VPN is IPSEC between hosts and works fine:

*vyos@vyos$ show vpn ipsec sa*
Connection                    State    Uptime    Bytes In/Out    Packets In/Out    Remote address    Remote ID    Proposal*
----------------------------  -------  --------  --------------  ----------------  ----------------  -----------  ---------------------------------------*
peer-187.X.X.X-tunnel-0  up       39m11s    6K/13K          117/234           187.X.X.X    N/A          3DES_CBC/HMAC_SHA1_96/MODP_1536*

iptables - all policies are ACCEPT and only rules are

vyos@vyos# sudo iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
VYATTA_PRE_DNAT_HOOK  all  --  anywhere             anywhere
LOG        all  --  200.X.X.X         194.X.X.3            /* DST-NAT-108 */ LOG level warning prefix "[NAT-DST-108] "
DNAT       all  --  200.X.X.X         194.X.X.3            /* DST-NAT-108 */ to:192.168.X.21

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
VYATTA_PRE_SNAT_HOOK  all  --  anywhere             anywhere
LOG        all  --  192.168.X.21        200.X.X.X         /* SRC-NAT-108 */ LOG level warning prefix "[NAT-SRC-108] "
SNAT       all  --  192.168.X.21        200.X.X.X         /* SRC-NAT-108 */ to:194.X.X.3

ESPs are commin on eth0 ok. TCP packets on eth0 gets decrypted and DNAT rule is applied, I see this communication on eth0 of VyOS, and DNAT rule in logs, but tcp goes only one way:

tcpdump -ni eth0 host 200.X.X.X or host 192.168.X.21 or host  194.X.X.3 or host 192.168.X.1
06:58:02.120064 IP 200.X.X.X.50625 > 194.X.X.3.443: Flags [S], seq 3712955870, win 14600, options [mss 1460,sackOK,TS val 2453686457 ecr 0,nop,wscale 7], length 0
06:58:05.121212 IP 200.X.X.X.50625 > 194.X.X.3.443: Flags [S], seq 3712955870, win 14600, options [mss 1460,sackOK,TS val 2453687208 ecr 0,nop,wscale 7], length 0
06:58:10.119714 IP 200.X.X.X.50625 > 194.X.X.3.443: Flags [R.], seq 3712955871, ack 0, win 0, length 0

Two sync’s and then reset, the other side gets no answer and resets with “Connection refused”
TCP packets on eth2 on VyOS flow ok, both sides try to talk but none reaches the other

tcpdump -ni eth2 host 200.X.X.X or host  194.X.X.3
07:02:01.530554 IP 200.X.X.X.51108 > 192.168.X.21.443: Flags [S], seq 3378212280, win 14600, options [mss 1460,sackOK,TS val 2453746309 ecr 0,nop,wscale 7], length 0
07:02:01.531361 IP 192.168.X.21.443 > 200.X.X.X.51108: Flags [S.], seq 3293980511, ack 3378212281, win 65160, options [mss 1460,sackOK,TS val 995534613 ecr 2453746309,nop,wscale 7], length 0
07:02:02.538879 IP 192.168.X.21.443 > 200.X.X.X.51108: Flags [S.], seq 3293980511, ack 3378212281, win 65160, options [mss 1460,sackOK,TS val 995535621 ecr 2453746309,nop,wscale 7], length 0
07:02:04.529170 IP 200.X.X.X.51108 > 192.168.X.21.443: Flags [S], seq 3378212280, win 14600, options [mss 1460,sackOK,TS val 2453747060 ecr 0,nop,wscale 7], length 0
07:02:04.530015 IP 192.168.X.21.443 > 200.X.X.X.51108: Flags [S.], seq 3293980511, ack 3378212281, win 65160, options [mss 1460,sackOK,TS val 995537612 ecr 2453746309,nop,wscale 7], length 0
07:02:06.538949 IP 192.168.X.21.443 > 200.X.X.X.51108: Flags [S.], seq 3293980511, ack 3378212281, win 65160, options [mss 1460,sackOK,TS val 995539621 ecr 2453746309,nop,wscale 7], length 0
07:02:09.531123 IP 200.X.X.X.51108 > 192.168.X.21.443: Flags [R.], seq 1, ack 1000986785, win 0, length 0
07:02:10.698941 IP 192.168.X.21.443 > 200.X.X.X.51108: Flags [S.], seq 3293980511, ack 3378212281, win 65160, options [mss 1460,sackOK,TS val 995543781 ecr 2453746309,nop,wscale 7], length 0

The same as on eth2 is tcp communication on eth1 of application server 192.168.X.21 so useless to paste it.

The ending -> up till eth2, communication is fine, where packets dissapear after that I cannot say. SNAT is not being applied. It does not appear in log file and obviously “monitor nat source rule 108” does show nothing. DNAT rule looks ok. What is interesting is that when I initiate telnet/netcat probe to 200.X.X.X from 192.168.X.21, SNAT is executed but I got connection timeout (maybe there is an issue on their side too). But this is my secondary problem, at the moment I cannot explain why SNAT is not executed and where are packets being routed.

On app server 192.168.X.21, static route for the remote IP is added via VyOS router eth2 interface:
200.X.X.0/24 via 192.168.X.1 dev eth1 proto static

On vyOS router, there is 220 table entry and routing:

ip xfrm policy:

src 194.X.X.3/32 dst 200.X.X.X/32
        dir out priority 367231 ptype main
        tmpl src 185.X.X.X dst 187.X.X.X
                proto esp spi 0x1a49cc29 reqid 7 mode tunnel
src 200.X.X.X/32 dst 194.X.X.3/32
        dir fwd priority 367231 ptype main
        tmpl src 187.X.X.X dst 185.X.X.X
                proto esp reqid 7 mode tunnel
src 200.X.X.X/32 dst 194.X.X.3/32
        dir in priority 367231 ptype main
        tmpl src 187.X.X.X dst 185.X.X.X
                proto esp reqid 7 mode tunnel

ip route get to 200.X.X.X
200.X.X.X via 185.X.X.1 dev eth0 table 220 src 194.X.X.3 uid 1003
    cache

It seems that response packets never reach eth0 and SNAT - even set to ‘any’ interface is not being done.
I tried to set log_martians but nothing is reported. I have tried to pass through kernel params with manual to see something suspicious, they are below. I played with

sudo sysctl -w net.ipv4.conf.all.log_martians=1
sudo sysctl -w net.ipv4.conf.eth0.log_martians=1
sudo sysctl -w net.ipv4.conf.eth0.route_localnet=1
sudo sysctl -w net.ipv4.conf.all.route_localnet=1
sudo sysctl -w net.ipv4.conf.eth0.accept_local=1
sudo sysctl -w net.ipv4.conf.all.accept_local=1

But no change, packets don’t get to eth0, no matter what I do. Any advice very welcome, I think I have all setup done properly.

In the end it appears that SNAT here does not get reported for the responses. Decrypting of the ESP packets show that correct ip headers are being sent anyway and the problem is on remote side. Everything works as expected then

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.