Multiple WAN interface - Source nat/outbound traffic not working

General questions
1

Hello all -

I have a pretty simple setup that I’m having a few problems with. I have 3 WAN interfaces, with ip addresses assigned all on the same subnet -

eth0 - wanip1
eth2 - wanip2
eth3 - wanip3

I have a solid /29 block, so it’s not like some of the hosting providers that for any additional IPs to use an interface route.

Here’s the problem I’m running into -
I have a static route defined for 0.0.0.0/0 to go out the gateway - wangateway1

If I have all 3 interfaces up, I can ping out of eth3 to the internet and get a reply (example, to 1.1.1.1)
if I try to use eth0 or eth2, I don’t get a reply, and the SNAT rules are not passing any traffic. I have a separate SNAT rule for the networks behind the vyos box

172.16.18.0/24
172.16.19.0/24
172.16.20.0/24

All configured to SNAT out of their respective WAN interfaces.

I’m not sure what I’m missing here.

Quick summary -

3 WAN interfaces
With all 3 enabled, only one can get ping replies from beyond the Default GW defined system-wide
Disable one, and the next in line starts working.

If anyone has any recommendations, please let me know.

Quick EDIT — I can connect to all 3 WAN interfaces with SSH, and it works.

TIA

  • reno
2

One more thing -

The internal (snat sources) ip addresses are all on eth1 vifs, connected to a trunk port, and all machines behind each vif can ping the external IP addresses.

3

You have a reverse path issue, most likely, and you’ve made this unnecessarily complex.

eth0 is your ‘feed’. That has wanip1, wanip2 and wanip3.
eth1 is 172.16.18.1/24
eth2 is 172.16.19.1/24
eth3 is 172.16.20.1/24

Once you fix this so you don’t have multiple conflicting routes, everything will start working.

4

You’re absolutely right. I have a tendency to make everything harder than it needs to be.

But, the reason I was trying to get this to work is so I can have a separate firewall for each IP. As far as I know (which isn’t too far) you can only apply firewalls to an interface, and not specifically to an IP. So, the only way I could think of to make that happen is to use multiple WAN interfaces.

If I’m wrong about the firewall bit I’ll be really happy, as it will make this whole thing far easier for setup and maintenance.

5

Well, I assume you’re doing dest nat from wanip1 to eth1, just set int eth eth1 firewall out name wanip1firewall

Otherwise it gets messy, yes.

6

I’ll give that a shot! Thanks for the direction!

Yes, there are several machines providing services on the same ports for different applications, so I need to keep this working on a per-wan-ip basis. If it were 1-to-1, it’d would be far easier, too. But, internally there are several different machines.

1 Like
7

That’s fine, it SHOULD work as you expect. Don’t forget to add firewall rule 1 accept related and established connections, too 8)

A really handy tip is sudo -i and then tcpdump -i eth1.200 -s0 -n -vvvvve to see the MAC addresses of all the packets on eth1 vif 200, which will help you with diagnosing other issues, too 8)