Dual wan, dual lan configuration but not load balance

General questions
1

Okay… been banging my head on the wall for at least 2 hours on this.

Setup:
WAN1: Cablemodem from ISP (eth0)
WAN2: Cablemodem from same ISP but business class. (eth1)

LAN1: 192.168.x.x (eth4) (connected to switch)
LAN2: 172.16.x.x (eth7) (connected to other switch)

What I want:
LAN1 <-> LAN 2 local
LAN1 → Outbound WAN1 (eth0)
LAN2 → Outbound WAN2 (eth1)

I’ve set up source nats as following:

 nat {
     source {
         rule 100 {
             destination {
                 address 0.0.0.0/0
             }
             outbound-interface eth0
             protocol all
             source {
                 address 192.168.69.0/24
             }
             translation {
                 address masquerade
             }
         }
         rule 200 {
             destination {
                 address 0.0.0.0/0
             }
             outbound-interface eth1
             protocol all
             source {
                 address 172.16.6.0/24
             }
             translation {
                 address masquerade
             }
         }
     }
 }

However, everything breaks unless there’s a default route on Vyos, and then it only fixes one of the two if I run a set protocol static 0.0.0.0/0 dhcp-interface eth[0/1] – which is kind of expected.

I’m running out of ideas and my google-fu is running dry.

2

Policy — VyOS 1.3.x (equuleus) documentation

In the “Multiple Uplinks” section is all you need.

In a example “OPTIONAL: Exclude Inter-VLAN traffic (between VLAN10 and VLAN11) from PBR” is little error, there shoulde be:

set policy route PBR rule 10 description 'VLAN10 <-> VLAN11 shortcut'
set policy route PBR rule 10 destination address '192.168.188.0/24'
set policy route PBR rule 10 set table 'main'
set policy route PBR rule 11 description 'VLAN10 <-> VLAN11 shortcut'
set policy route PBR rule 11 destination address '192.168.189.0/24'
set policy route PBR rule 11 set table 'main'

or

set firewall group network-group local-nets network '192.168.188.0/24'
set firewall group network-group local-nets network '192.168.189.0/24'
set policy route PBR rule 10 description 'VLAN10 <-> VLAN11 shortcut'
set policy route PBR rule 10 destination group network-group 'local-nets'
set policy route PBR rule 10 set table 'main'
3

Create tables and define default gateways for both tables

set protocol static table 10 route  0.0.0.0/0 dhcp-interface eth0
set protocol static table 20 route  0.0.0.0/0 dhcp-interface eth1

Create Policies

set policy route PBR rule 10 set table '10'
set policy route PBR rule 10 description 'Route LAN1 traffic to table 10'
set policy route PBR rule 10 source address '192.168.69.0/24'
set policy route PBR rule 20 set table '20'
set policy route PBR rule 20 description 'Route LAN2 traffic to table 20'
set policy route PBR rule 20 source address '172.16.6.0/24'

# Allow LAN1-LAN2 communication through main table
set policy route PBR rule 30 description 'LAN1 shortcut'
set policy route PBR rule 30 destination address '192.168.69.0/24'
set policy route PBR rule 30 set table 'main'
set policy route PBR rule 40 description 'LAN2 shortcut'
set policy route PBR rule 40 destination address '172.16.6.0/24'
set policy route PBR rule 40 set table 'main'

Assing policy to interfaces:

set interfaces ethernet eth4 policy route 'PBR'
set interfaces ethernet eth7 policy route 'PBR'

Also, I guess that NAT rules should be present in your case

set nat source rule 10 description "Nat to Internet through WAN1"
set nat source rule 10 outbound-interface eth0
set nat source rule 10 translation address masquerade
set nat source rule 20 description "Nat to Internet through WAN2"
set nat source rule 20 outbound-interface eth1
set nat source rule 20 translation address masquerade
2 Likes
4

Okay, applied… but now I lost inter-lan, wouldn’t the rule 10/20 be executed before the rule 30/40, causing stuff to go out the WAN before it hits the main table?

5

Indeed, PBR rules 30, 40 should come before 10 , 20

6

Yes, I’m sorry for that. My mistake!
Definitely I wrote rules in wrong order. Rules 30 and 40 should be executed before rules 10 and 20!

7

Well, good news, I’m over that hump, now I’m trying to get some ipsec goodness over to vultr so that I can share my BGP ip space… from what it sounds like is that I need to use vti because of pbr, right?

8

Alright, I’m adding to my configuration.

I have a VULTR vps that’s advertising my V4/24 space. I’ve carved out a /27, BGP announces fine, however, I cannot for the life of me get the inbound traffic to the public IPs to traverse. I’m to the point where I think it’s something, really, really simple that I’m missing because I’m so focused somewhere else.

The only thing I haven’t done is put everything on a NAT, but, if I’m routing the IPs publicly over wireguard, I shouldn’t need to NAT it, right?

I can ping from wireguard to wireguard, so that’s up, so, wireguard config looks good.

On-Premise

vyos@vyos# show | strip-private
 firewall {
     name OUTSIDE_LOCAL {
         rule 10 {
             action accept
             description "Allow established/related"
             state {
                 established enable
                 related enable
             }
         }
         rule 20 {
             action accept
             description WireGuard_IN
             destination {
                 port 51820
             }
             log enable
             protocol udp
             source {
             }
         }
     }
 }
 interfaces {
     ethernet eth0 {
         address dhcp
         description "uplink to twc resi"
         hw-id 3c:ec:ef:6e:05:2c
     }
     ethernet eth1 {
         address dhcp
         description "uplink to twc biz"
         firewall {
             local {
                 name OUTSIDE_LOCAL
             }
         }
         hw-id 3c:ec:ef:6e:05:2d
     }
     ethernet eth2 {
         hw-id 3c:ec:ef:6e:05:2e
     }
     ethernet eth3 {
         hw-id 3c:ec:ef:6e:05:2f
     }
     ethernet eth4 {
         address xxx.xxx.69.254/24
         hw-id 3c:ec:ef:6e:05:30
         policy {
             route PBR
         }
     }
     ethernet eth5 {
         hw-id 3c:ec:ef:6e:05:31
     }
     ethernet eth6 {
         address xxx.xxx.6.1/24
         hw-id 3c:ec:ef:6e:08:12
         policy {
             route PBR
         }
     }
     ethernet eth7 {
         hw-id 3c:ec:ef:6e:08:13
     }
     loopback lo {
     }
     wireguard wg01 {
         address xxx.xxx.203.130/27
         description VPN-to-wg02
         peer to-wg02 {
             address xxx.xxx.145.166
             allowed-ips xxx.xxx.0.0/0
             port 51820
             public-key [strip-private didnt strip this]
         }
         policy {
             route PBR
         }
         port 51820
         private-key xxxxxx
     }
 }
 nat {
     source {
         rule 100 {
             outbound-interface eth0
             protocol all
             translation {
                 address masquerade
             }
         }
         rule 200 {
             outbound-interface eth1
             protocol all
             translation {
                 address masquerade
             }
         }
     }
 }
 policy {
     route PBR {
         rule 5 {
             destination {
                 address xxx.xxx.203.128/27
             }
             set {
                 table main
             }
         }
         rule 10 {
             destination {
                 address xxx.xxx.69.0/24
             }
             set {
                 table main
             }
         }
         rule 20 {
             destination {
                 address xxx.xxx.6.0/24
             }
             set {
                 table main
             }
         }
         rule 30 {
             set {
                 table 10
             }
             source {
                 address xxx.xxx.69.0/24
             }
         }
         rule 40 {
             set {
                 table 20
             }
             source {
                 address xxx.xxx.6.0/24
             }
         }
         rule 50 {
             set {
                 table 30
             }
             source {
                 address xxx.xxx.203.128/27
             }
         }
     }
 }
 protocols {
     static {
         table 10 {
             route xxx.xxx.0.0/0 {
                 dhcp-interface eth0
             }
         }
         table 20 {
             route xxx.xxx.0.0/0 {
                 dhcp-interface eth1
             }
         }
         table 30 {
             route xxx.xxx.0.0/0 {
                 interface wg01 {
                 }
                 next-hop xxx.xxx.203.129 {
                 }
             }
         }
     }
 }
 service {
     dhcp-server {
         global-parameters "option space ubnt;"
         global-parameters "option ubnt.unifi-address code 1 = ip-address;"
         global-parameters "class &quot;ubnt&quot; {"
         global-parameters "match if substring (option vendor-class-identifier, 0, 4) = &quot;ubnt&quot;;"
         global-parameters "option vendor-class-identifier &quot;ubnt&quot;;"
         global-parameters "vendor-option-space ubnt;"
         global-parameters "}"
         shared-network-name home {
             authoritative
             subnet xxx.xxx.69.0/24 {
                 default-router xxx.xxx.69.254
                 dns-server xxx.xxx.1.1
                 lease 3600
                 range 100 {
                     start xxx.xxx.69.100
                     stop xxx.xxx.69.200
                 }
                 subnet-parameters "option ubnt.unifi-address unifi.powertogrow.com;"
             }
         }
         shared-network-name rack {
             authoritative
             subnet xxx.xxx.6.0/24 {
                 default-router xxx.xxx.6.1
                 dns-server xxx.xxx.1.1
                 lease 3600
                 range 100 {
                     start xxx.xxx.6.100
                     stop xxx.xxx.6.200
                 }
             }
         }
     }
     dns {
         forwarding {
             allow-from xxx.xxx.69.0/24
             allow-from xxx.xxx.6.0/24
             dnssec process-no-validate
             listen-address xxx.xxx.6.1
             listen-address xxx.xxx.69.254
             name-server xxx.xxx.1.1
             name-server xxx.xxx.2.1
             name-server xxx.xxx.8.8
             name-server xxx.xxx.4.4
             name-server xxx.xxx.0.1
         }
     }
     ssh {
         port 2
         port 22
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     conntrack {
         modules {
             ftp
             h323
             nfs
             pptp
             sip
             sqlnet
             tftp
         }
     }
     console {
         device ttyS0 {
             speed 115200
         }
     }
     host-name vyos
     login {
         user vyos {
             authentication {
                 encrypted-password xxxxxx
                 plaintext-password xxxxxx
             }
         }
     }
     name-server xxx.xxx.1.1
     ntp {
         server 0.pool.ntp.org {
         }
         server 1.pool.ntp.org {
         }
         server 2.pool.ntp.org {
         }
     }
     syslog {
         global {
             facility all {
                 level info
             }
             facility protocols {
                 level debug
             }
         }
     }
 }

On VULTR

star@vyos# show | strip-private
 interfaces {
     ethernet eth0 {
         address xxx.xxx.145.166/23
     }
     ethernet eth1 {
         address xxx.xxx.96.6/20
     }
     loopback lo {
     }
     wireguard wg01 {
         address xxx.xxx.203.129/24
         description VPN-to-wg01
         peer to-wg02 {
             address xxx.xxx.228.72
             allowed-ips xxx.xxx.0.0/0
             port 51820
             public-key [strip, grep, you know it]
         }
         port 51820
         private-key xxxxxx
     }
 }
 protocols {
     bgp {
         address-family {
             ipv4-unicast {
                 network xxx.xxx.203.128/27 {
                 }
             }
         }
         local-as 397898
         neighbor xxx.xxx.169.254 {
             address-family {
                 ipv4-unicast {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             disable-connected-check
             ebgp-multihop 2
             password xxxxxx
             remote-as 64515
             timers {
                 holdtime 30
                 keepalive 10
             }
         }
         neighbor wg01 {
             interface {
                 remote-as internal
             }
         }
         parameters {
         }
     }
     static {
         route xxx.xxx.0.0/0 {
             next-hop xxx.xxx.144.1 {
             }
         }
         route xxx.xxx.203.128/27 {
             interface wg01 {
             }
             next-hop xxx.xxx.203.130 {
             }
         }
         route xxx.xxx.6.0/24 {
             interface wg01 {
             }
         }
         route xxx.xxx.69.0/24 {
             interface wg01 {
             }
         }
     }
 }
 service {
     ssh {
         listen-address xxx.xxx.145.166
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     conntrack {
         modules {
             ftp
             h323
             nfs
             pptp
             sip
             sqlnet
             tftp
         }
     }
     console {
         device ttyS0 {
             speed 115200
         }
     }
     host-name vyos
     login {
         user star {
             authentication {
                 public-keys xxxx@xxx.xxx {
                     key xxxxxx
                     type ssh-rsa
                 }
             }
         }
         user vyos {
             authentication {
                 encrypted-password xxxxxx
             }
         }
     }
     name-server xxx.xxx.1.1
     ntp {
         server time1.vyos.net {
         }
         server time2.vyos.net {
         }
         server time3.vyos.net {
         }
     }
     syslog {
         global {
             facility all {
                 level info
             }
             facility protocols {
                 level debug
             }
         }
     }
 }
9

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.