Unstablities loadbalancing

I recently got an second fiber line, now I am trying to setup load balancing.
I had it setup and working at-least from an inside perspective i could get to the outside world and i could get different IP’s at each machines but for some reason all connections from outside to inside seems to fail well except SSH.
I could not ping either of the wan IP’s and HTTP/HTTPS port forwarding with nat destination seemed to not work.
I made all NAT Destination rules for either if my interfaces.

But i cant find out why it was not working, and to be honest the same with my tunnelbroker would not work with wan load-balancing on.
Does anyone have any ideas what can be the cause?

Hi,

did you take a look at this? Load-balancing — VyOS 1.4.x (sagitta) documentation

maybe there are some helpful hints.
If you can’t ping the wan IPs have you check the firewall rules?
You can tcpdump the traffic and look if its fit your nat or firewall config.

For more help, i need some more infos. a little picture of your structure and a dump of the related config.

show configuration commands | strip-private

Hi Rob,
Yea i even followed most of it. and i was missing some things that i found out how to do from an forum post (forgot the health checks)
Due to the length i have put it in an pastebin if that is fine with you, here is the link to it.

Vif 2 is my Cable connection which goes in an untagged port in my switch, Vif 300 is my fiber connection, my vyos is in a trunk to switch which has all the connections going thru it.

I try your setup in my lab. The DNAT stuff worked fine for me.
Can you monitor your traffic and check if it look like you want?

tcpdump -i eth0.2 -nnn host IPADDRESS
tcpdump -i eth1.300 -nnn host IPADDRESS

you can filter the destination or source IPADDRESS or more filter here

you also can take a look at the internal interfaces, where the natted packet should go.

to show the status of nat you can look here:

show nat destination translations

Hi Rob,
I just did another scan and TCP Dump,
It seems i can successfully ping 1 of the 2 interfaces and do an port scan on it but not the other.
Edit:
ping to wan 1 is successfull from some ip’s, ports dont seem to be open for that ip even if the dnat rules exist
Ping to wan 2 is successful from all ip’s, all ports seem to be reachable.

The fact you can only ping from some IPs, is strange. did your provider block something?
When you dump your “wan 1” connection did you see the icmp traffic?

at the moment i’m clueless.

Okay a bit more clear sympthons and i can PM you both IP’s so you can also test.
Going to be more clear to
Vyos runs in a VM on Proxmox
Currently vyos is connected to 2 bridges
1 for lan vlans
1 for wan vlans
Both cables go as trunk to my switch
Wan 1 is an Docsis modem in bridge mode
Wan 2 is a SFP module from my ISP direct on fiber.

i just tried the following
Logged in on my server using mobile data.
ping to wan 1 IP, Got response
Curl to Wan 1 IP, Got response

5 seconds later
Curl to wan 2 IP, No response
Curl to wan 2 IP, No response
I think it’s either the routing table on vyos, or the routing table on the host, but my guess is going to vyos.

Edit: Also soon the second lan bridge will be nuked and replaced with an dedicated dual SFP card due to an upgrade to dual fiber.
Also sorry if it was not fully clear in the begin (at least i feel i wasn’t fully clear)

yes, and if you have a little drawing or picture of you structure.

Here us a quick overview of the diagram

PM has been send.

strange, i can ping both IPs but got no open ports on 2x.x.x.112

i got one hint from a colleague, in 1.3 rolling wan loadbalancing is partial broken.
But i found no related issue to your problem.

Which version you have? I will retest later with the latest rolling.

show version

Yea its wierd, and what i also noticed is if you ping 31.x.x.x once it wouldn’t let me ping 2.x.x.112 anymore
Version is VyOS 1.3-rolling-202010070117

i retest my test setup with VyOS 1.3-rolling-202010140146 with no problems.

i also noticed is if you ping 31.x.x.x once it wouldn’t let me ping 2.x.x.112

i can’t reproduce this.

The last thing i can recommend is to the test your DNAT rules without the wan loadbalancing configuration.

Without Wan Load Balancing DNAT works great
Actually i just tried and when i pinged 2xx it worked, but if i try to ping 31 right after that i get no response
see attached vid

Just updated to latest rolling
Tried this
Ping 217.* got response
ping 31.* got no response

Curl 31.* Got response
Curl 217.* got no response

For now Wan Loadbalancing is disabled as one of my wans is gone, when i get my second uplink back i will test again and see if the problems are gone then.
if not i might go for an stable release.

If you want that response was from both interfaces you need “local” PBR.
https://phabricator.vyos.net/T439
And ⚓ T2747 "enable-local-traffic" has no effect in load-balancing to redirect local traffic
So as forkaround.

set protocols static table 100 interface-route 0.0.0.0/0 next-hop-interface eth0.2
set protocols static table 101 interface-route 0.0.0.0/0 next-hop-interface eth1.300

And scripts

#!/usr/bin/env bash

my_eth0_ip=$(ip addr show eth0.2 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)
my_eth1_ip=$(ip addr show eth1.300 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)

ip rule add from $my_eth0_ip lookup 100
ip rule add from $my_eth1_ip lookup 101

Or set ip addresses manual (in ip rule).

Thank you, I will deploy this when i get my second wan back.

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.