NAT is really not what you want here. Source-NAT is one of the last operations performed on a packet before it’s routed out an interface. The decision to route out that interface is made earlier. Therefore, I don’t think you can source-NAT your way out an interface.
Also, this is not VyOS specific. Other platforms behave similarly (i.e. Cisco).
What you need to do is coerce packets or flows to be routed out a particular interface. Whether source-NAT is applied is irrelevant (except if you must do so because it’s a public Internet link).
What you want is Policy Based Routing (PBR), or just static route out the relevant interface for specific IP prefixes.
I’ve had not configured or verified PBR in VyOS (but have done so using underlying Linux & commercial routers), so I thought I’d overcomplicate my home network and simulate a second gateway path to test PBR. I configured things as Carl documented in 510 Software Group and it worked perfectly. I policy routed traffic destined to 8.8.8.8 out my second internet link.
This is all the config to it (I’ve added comments to aid in understanding):
(note I’m using the nightly lithium builds for this testing)
vyos@vygw01:~$ show configuration commands | egrep '(policy|pbr-test|table 1)'
# just a standard firewall group object
set firewall group network-group pbr-test network '8.8.8.8/32'
# traffic matching network-group 'pbr-test' will cause VyOS to use routing table '1' for route lookups
set policy route pol-route-pbr-test rule 10 destination group network-group 'pbr-test'
set policy route pol-route-pbr-test rule 10 set table '1'
# for route table '1', set default route via second ISP path
set protocols static table 1 route 0.0.0.0/0 next-hop '192.168.253.1'
# apply to *inside* interface. Not either of your ISP paths
set interfaces ethernet eth0 vif 10 policy route 'pol-route-pbr-test'
I have done several combinations of policy routing rules and continue to fail. I am not sure, but it seems that this is not compatible with a rule-based load-balancing failover setup (as described in the Vyatta-HA_R6.4_v01.pdf ).
Did you test your setup with a load-balancing failover setup? I have not tried load-balancing failover based on weight, but I was wondering if this would make any difference.
PBR and wan-load-balacing are mutually exclusive as they both use iptables MARKs to properly identify and route packets and they both use a set of specific routing tables.
You could set some exclude rules in load-balacing to have some traffic fallback to PBR but that’s about it with current VyOS (Lithium build).
Well, the original question was how to improve (or prioritize) VoIP/voice quality, with the assumption that both data and voice would run on one connection.
However, I created another thread in which I asked about load-balancing, and then I asked about directing traffic based on protocol or IP – specifically, in relation to VoIP. Since these two were related, someone that was responding to both threads ended up posting answers here that related to both. And that is how we ended up mixing the threads, or you could say it was just the evolution of the thread from one possible solution to another more efficient.
[quote=‘kouak’]PBR and wan-load-balacing are mutually exclusive as they both use iptables MARKs to properly identify and route packets and they both use a set of specific routing tables.
You could set some exclude rules in load-balacing to have some traffic fallback to PBR but that’s about it with current VyOS (Lithium build). [/quote]
I suspected as much. I was wondering about the exclude portion, but have not gotten to it, yet. To exclude an IP, all I need to do is add ! in front of the IP, correct?
One suggestion would be to configure PBR in isolation without WAN load balancing and get something working before trying to get both working. You’ll be chasing your tail as to why it’s not working in the config isn’t correct.
So, I tried using the exclude command, but my config would still route through the main outbound interface. I tried changing the rule order. Originally, I had my first rule setup to send traffic out eth0, and my second rule was to send it out eth1…
set load-balancing wan rule 10 inbound-interface eth2
set load-balancing wan rule 10 interface eth0
set load-balancing wan rule 20 inbound-interface eth2
set load-balancing wan rule 20 interface eth1
set load-balancing wan rule 110 exclude
set load-balancing wan rule 110 inbound-interface eth2
set load-balancing wan rule 110 destination address 75.75.75.75/32
…so I moved this further down, to let the system process it after it had processed my exclusion.
set load-balancing wan rule 10 exclude
set load-balancing wan rule 10 inbound-interface eth2
set load-balancing wan rule 10 destination address 75.75.75.75/32
set load-balancing wan rule 110 inbound-interface eth2
set load-balancing wan rule 110 interface eth0
set load-balancing wan rule 120 inbound-interface eth2
set load-balancing wan rule 120 interface eth1
That did not change the result, I was still not able to route the proper interface. I changed it back to the original order, and every once in a while I would see the proper route being used. This happened at random, because I could not replicate it by changing something, rather by simply sending the same command over and over again, until it happened (or not).
I was looking at that config from the five ten group and I only see the portion the relates to the one DSL link, but I don’t see the rest of the setup. Did they ever publish the rest of their setup? Creating a single routing policy sure works fine, but is the rest of the config setup with VyOS commands, or is it based on scripts like the one at the bottom of that page?
It seems like the best solution would be to input a ‘set load-balancing wan’ rule for every IP that I want to re-route traffic for, and that would be the way to go for now. I think I am getting close to a more organized solution, though. I would really like to get the network objects working in the future. Thanks for the help with this, I think I have gained quite an amount of understanding of VyOS from the responses here.
What you’ve done here is remove wan load balacing marking for traffic arriving on interface eth2 with 75.75.75.75/32 as destination.
This traffic will be left alone by the load balancing rules and will fall back to the kernel main routing table (visible by issuing a “sudo ip route” command).
Most likely, you have 2 default routes in your kernel main routing table (since you have 2 wan links). What you want to do now, is force 75.75.75.75/32 to leave through a specific wan link and this is done by adding a static route to 75.75.75.75/32.
# set protocols static route 75.75.75.75/32 next-hop WAN-GATEWAY
I believe I had a static route setup for the IP and it did not work, but I will have to try it again, since you have confirmed that the way I had set it up was correct. I just realized what could have been causing my problem with the route not sticking for long. I had setup an IP address that does not reply as part of my ‘test ping’ for load-balancing (to determine if that interface’s route was active) and this was probably causing the system to automatically fail that interface and force the use of eth0.
However, I think I just stumbled on something better than what I was trying to accomplish. I was trying to copy that config from the 510group, but their config is overcomplicated by introducing the use of shell scripting – which is not appealing for someone without much understanding of shell scripting, or any other scripting. What I just realized is that by adding my IP addresses as a rule in load-balancing, I have a built-in failover feature in place, regardless of which interface I use for my route. This is the setup I have right now:
wan {
interface-health eth0 {
failure-count 5
nexthop 10.0.2.1
success-count 1
test 10 {
resp-time 5
target 192.168.0.101
ttl-limit 1
type ping
}
}
interface-health eth1 {
failure-count 4
nexthop 10.10.0.1
success-count 1
test 10 {
resp-time 5
target 192.168.0.104 #this does not reply, so it keeps failing
ttl-limit 1
type ping
}
}
rule 9 {
destination {
address 75.75.75.75
}
inbound-interface eth2
interface eth1 {
weight 10
}
protocol all
}
rule 11 {
inbound-interface eth2
interface eth0 {
weight 1
}
protocol all
}
rule 20 {
destination {
}
inbound-interface eth2
interface eth1 {
weight 1
}
protocol all
}
}
I have been testing this and it works in both directions, when eth1 is down, my traffic for 75.75.75.75 will use eth0. And when eth0 is down, my traffic for eth0 will use eth1 as a backup. So I don’t need a script to tell the system to remove routes or anything, it is already built into my load-balancing failover setup.
This is the setup in my test environment, but if I was to use it in production, I would definitely start with a much higher number rule (like 100) to leave enough room for exceptions. I still need to fix my problem with the IP target that keeps causing my interface to fail, so I need to attache a new device with another IP address and use it to see how things change. This way I can go back to the drawing board and start testing with network objects, again. Maybe this time it will work.
So, this goes back to my VyOS and VoIP question, but a little different. I know how to setup a traffic-policy to prioritize VoIP (or at least as far as the guide from 510 Software Group has shown me). But I am still not sure if my policy is all that I need, or if I need to do some configuration of the packets before they enter the router (on the switch), or as they enter the router (via PBR). As the traffic-policy only applies on egress, I am still wondering if I need to use PBR to mark packets with DSCP 46 or 26, whichever may apply so that my traffic-policy can then recognize them as voip-rtp or voip-sip (match settings in the policy).
I am currently running my VoIP and Data on the same connection, and all I have is the traffic-policy, but I am neither marking them via PBR, nor do I have a voic VLAN setting in my switches. So I want to know if my policy is pretty much useless, or is it doing something?
Even though I will dedicate one separate connection to VoIP in the very near future, I would still like to have the two connections configured properly, in case there is downtime on either one, and I will need to use the other as a backup.
Do you need QoS for this? Does it work without? When the packet leaves your router / network I don’t think the rest of the internet trust your marking of the IP packets.
I don’t have any problems with VoIP without QoS, but the LAN must be cleaned up properly.
You don’t strictly need DSCP set on any switch or router prior to your edge. DSCP does make life easier when classifying traffic as bulk, low latency etc. But you can always do it on other criteria (src/dst port, src/dst IP).
Egress QOS is about bandwidth management - in the event of congestion, what packets should be dequeued & transmitted first. Ideally that is small packets (TCP acks, SYN, DNS etc) and VOIP traffic. You want to avoid ever exceeding your egress throughput by overtransmitting (e.g. your physical layer is 10M or 100M, but you are provided 2M. You can easily oversend and fill remote buffers). So part of your QoS strategy might be to reserve bandwidth for important stuff. Another part might be to ensure some traffic never exceeds a certain throughput.
Others say never share a WAN link for VOIP and non-VOIP - a suggestion I whole-heartedly recommend after dealing with voip quality issues far too many times
Well, we have a lot of equipment that makes use of the bandwidth and is constantly sending and receiving, so I would definitely like the VoIP to have priority. Once out in the world, that is a different story. I have been running the VoIP on its own router (Cisco AE6900) and dedicated fiber connection, but have had a lot of problems and the downtime is ridiculous. After going back and forth with support (the vendor), I finally decided to move them to my VyOS router and it was working okay for the past day and a half. I am not sure if there other minor issues with voice quality, but the system is actually functional and receiving calls. Given all the issues that we have had, I am only able to assume that the problems reside with the vendor, rather than our connection, as we have moved from one router to another a few times, and the problem persists. We now moving to Five9 in a week or so, and I want to get my network ready for this and do anything possible to make it work without a hiccup.
According to one of the guides I was looking at, IP Phone and Asterisk set DSCP-46 automatically (?), so maybe the policy does not need PBR? I guess packets are already marked and all I need is the policy to recognize these markings? Can someone confirm this?
