My first time vyos experience!

General questions
, , ,
1

Hey, I have been using router os for a few years on a RB450GX4. The device was quite weak for my needs so I upgraded to a x64 machine(N305, 6x i226 nics). I tried pfsense, opnsense and untangled and didn’t like them too much and now I have finally almost finished setting up vyos. So far I do really like it but there is stuff that can be improved. I made small notes when setting it up about the things I’d like to see!

  1. I have a bad_ipv6 ipv6-network-group. These addresses should be a part of that group but vyos tells me they are invalid and does not add them to this group.
network "::ffff:0.0.0.0/96"
network "::224.0.0.0/100"
network "::127.0.0.0/104"
network "::255.0.0.0/104"

I believe this is a bug

  1. Is there a way to inverse a filter? For example, I’d like to drop all traffic from WAN that has not been DNAT’ed. So for this, I should have a firewall rule that looks like
set firewall ipv4 forward filter rule 1 action drop
set firewall ipv4 forward filter rule 1 description "Drop all from WAN not DSTNATed"
set firewall ipv4 forward filter rule 1 connection-status nat !destination

but I am not sure if I can apply inverse filters like this? I did see docs contexts in some cases where it suggested I could do !<resource> but then I don’t see it in other places.

  1. Similar to 2, I could merge this into a single rule if inverse filters are possible.
                rule 14 {
                    action "accept"
                    inbound-interface {
                        interface-name "wg0"
                    }
                    source {
                        group {
                            network-group "personal_vpn"
                        }
                    }
                }
                rule 15 {
                    action "drop"
                    inbound-interface {
                        interface-name "wg0"
                    }
                }

INTO

                rule 14 {
                    action "drop"
                    inbound-interface {
                        interface-name "wg0"
                    }
                    source {
                        group {
                            network-group "!personal_vpn"
                        }
                    }
                }
  1. This can be a small improvement. I’ll prefer the ability to select interface groups when configuring SNAT rules. I had to add multiple rules (1 for each WAN)
set nat destination rule 2 inbound-interface eth1
set nat destination rule 2 inbound-interface pppoe0
set nat destination rule 2 inbound-interface wg1

If it was possible to select interface groups here I could do,

set nat destination rule 2 inbound-interface interface-group WAN
  1. Similar problems as 4. It’s not possible to select interface groups in load-balancing rules.
set load-balancing wan rule 1 inbound-interface eth1
set load-balancing wan rule 1 inbound-interface pppoe0
  1. monitor and log commands can use a lot of features.

6a. A command like, monitor wan-load-balance similar to show wan-load-balance.
6b. monitor log <line-count>

I have more configuration left to do right now, mostly related to ipv6, monitoring and selective routing of traffic from some hosts over configured wireguard VPN. I’ll try to do that in the next few days

2

From other threads, It looks like this may not be possible today

3

  1. If ipv6 address are bad_ipv6, as you said, why they should be allowed?

  2. No, so far it’s not possible inverted selection for connection-status. You can submit a feature request in https://vyos.dev

  3. That It’s allowed! Maybe we should extend description/helpers in vyos cli

vyos@default-log# run show config comm | grep "forward filter"
set firewall ipv4 forward filter rule 10 action 'drop'
set firewall ipv4 forward filter rule 10 inbound-interface interface-name 'eth3'
set firewall ipv4 forward filter rule 10 log 'enable'
set firewall ipv4 forward filter rule 10 source group network-group '!NG01'

  1. Interface groups in NAT rules: not supported, but I’ve already created feature request for it: ⚓ T5643 NAT - Allow interface groups on nat rules

  2. As you have already discoverd, wan load balance still used old code, so it’s really hard to implement new things over there.

  3. Not getting exact idea on what you need

4
  • If ipv6 address are bad_ipv6, as you said, why they should be allowed?

Sorry, I misspoke. I want to drop that traffic and I can’t add those addresses to the ipv6 network group.

No, so far it’s not possible inverted selection for connection-status. You can submit a feature request in https://vyos.dev

I shall do that in a few days.

That It’s allowed! Maybe we should extend description/helpers in vyos cli

Yes please! In some places I did see x.x.x.x/x and !x.x.x.x/x in description but I didn’t see it here and I assumed it was not possible.

Thank you!

Not getting exact idea on what you need

When testing or just in general, I want to easily monitor wan load balance state(if it’s able to reach the gateway, if there are any pings that failed etc). It’ll be great to have a monitor wan-load-balance command for it.

For 6b, I would like an option to select the number of lines to show similar to show log 500 which shows 500 lines from the end. So monitor log 500 shows last 500 lines and follows the stream.

5

I think some preprocessor is incorrect in VyOS (like node.def level) since this is a valid IPv6 address (well at least in writing):

vyos@vyos# set firewall group ipv6-network-group V6_WAN network ::ffff:0.0.0.0/96

  Error: ::ffff:0.0.0.0/96 is not a valid IPv6 prefix
  
  
  
  Invalid value
  Value validation failed
  Set failed

Writing last 32 bits as IPv4 is a thing, they should be converted into hex before evaluated by the application.

That is normally taken care of by having a default-action of drop and then only allow for the flows you want to allow for.

As you can see here Netfilter hooks - nftables wiki the DNAT occurs in PREROUTING which means that the firewall rule in filter forward should allow for the DNATed value.

Example:

Your WAN address is 10.0.0.1 and you DNAT incoming TCP80 to DMZ server 192.168.0.100.

Then the firewall rule to allow for the DNATed traffic would allow dstip 192.168.0.100 (and port tcp80).

The non-DNATed traffic would then automatically be dropped by default-action:drop.

Again this is taken care of by default-action OR if you do something like this:

rule 14 inbound: wg0, source-group:!personal_vpn, action:drop
rule 15 inbound: wg0, action:allow

Since nftables (just like previous iptables) is a top-down/first-match firewall you can define the more specific first (rule 14) which means that whatever rule14 didnt match will be handled by rule15.

Good idea and it seems like @n.fort already created a commit regarding this :slight_smile:

@n.fort If Im not mistaken isnt keepalived used for this?

And if so and even if keepalived doesnt supports interface-groups couldnt the preprocessor of VyOS take care of that?

So a “inbound-interface-group LAN” would be expanded into how many “inbound-interface xxx” is needed?

Also are there alternative methods to perform an egress “WAN-balancing” now with the new firewall frontend?

The main issues are often the loadbalancing itself (like roundrobin or least-connections and such) but also “stickyness” that is a packet who belongs to a flow where WAN2 was selected should continue to use that as egress otherwise the firewall itself with its connection tracking will get sad (and drop packets if they suddently are sent to WAN1).

Yeah me neither fully understand the request.

6

  1. Got it!

  2. I do have a default-drop rule. I still would have to use a inverse filter or use 2 rules to do this because of how I wrote my firewall rules. (i’ll make a separate post for it)

  3. It was just some QoL stuff, we can skip that :grimacing: