Firewall Rules in a VRRP setup not working as expected

jg75996 · December 10, 2020, 6:57pm

Hello everyone,

I need some assistance with a setup that I am trying as a proof of concept. In GNS3, I have a setup where there is a Cisco router, a VyOS router, a switch, and 3 network endpoints: a PC, a phone, and a network appliance.

Networks in use are the following:
Default VLAN1 for the PCs - 192.168.8.0/22
VLAN 100 for the network appliances - 10.100.8.0/29
VLAN 700 for the phones - 172.16.8.0/23

The Cisco router and VyOS router are setup in a VRRP configuration.

There is an acl-list defined on the Cisco router to prevent the network appliances from communicating with other devices on the network. The relevant parts of the config are as follows:

interface FastEthernet0/0.100
description Restricted_Devices_Network
encapsulation dot1Q 100
ip address 10.100.8.5 255.255.255.248
ip access-group Filter_Restricted_Devices in
vrrp 60 description Restricted_Devices_Network
vrrp 60 ip 10.100.8.1
vrrp 60 priority 110

interface FastEthernet0/0.700
encapsulation dot1Q 700
ip address 172.16.8.2 255.255.255.0
vrrp 20 description VoIP
vrrp 20 ip 172.16.8.1
vrrp 20 priority 110
!
interface FastEthernet0/1
ip address 192.168.8.2 255.255.248.0
duplex auto
speed auto
vrrp 30 description Branch LAN
vrrp 30 ip 192.168.10.16
vrrp 30 priority 110

ip access-list extended Filter_Restricted_Devices
deny ip 10.100.8.0 0.0.0.7 192.168.0.0 0.0.255.255
deny ip 10.100.8.0 0.0.0.7 172.16.0.0 0.0.255.255
permit ip any any

On the VyOS router, the relevant parts of the config are as follows:

set firewall all-ping ‘enable’
set firewall broadcast-ping ‘disable’
set firewall config-trap ‘disable’
set firewall group network-group Internal_Networks network ‘192.168.0.0/16’
set firewall group network-group Internal_Networks network ‘172.16.0.0/16’
set firewall ipv6-receive-redirects ‘disable’
set firewall ipv6-src-route ‘disable’
set firewall ip-src-route ‘disable’
set firewall log-martians ‘enable’
set firewall name Prevented_Networks default-action ‘accept’
set firewall name Prevented_Networks enable-default-log
set firewall name Prevented_Networks rule 5 action ‘reject’
set firewall name Prevented_Networks rule 5 destination group network-group ‘Internal_Networks’
set firewall name Prevented_Networks rule 5 log ‘enable’
set firewall receive-redirects ‘disable’
set firewall send-redirects ‘enable’
set firewall source-validation ‘disable’
set firewall syn-cookies ‘enable’
set firewall twa-hazards-protection ‘disable’
set high-availability vrrp group BranchLAN interface ‘eth2’
set high-availability vrrp group BranchLAN priority ‘50’
set high-availability vrrp group BranchLAN rfc3768-compatibility
set high-availability vrrp group BranchLAN virtual-address ‘192.168.10.16/22’
set high-availability vrrp group BranchLAN vrid ‘30’
set high-availability vrrp group Network_Appliances interface ‘eth3’
set high-availability vrrp group Network_Appliances priority ‘50’
set high-availability vrrp group Network_Appliances rfc3768-compatibility
set high-availability vrrp group Network_Appliances virtual-address ‘10.100.8.1/29’
set high-availability vrrp group Network_Appliances vrid ‘60’
set high-availability vrrp group VoIP interface ‘eth1’
set high-availability vrrp group VoIP priority ‘50’
set high-availability vrrp group VoIP rfc3768-compatibility
set high-availability vrrp group VoIP virtual-address ‘172.16.8.1/23’
set high-availability vrrp group VoIP vrid ‘20’
set high-availability vrrp sync-group Main member ‘Firewall’
set high-availability vrrp sync-group Main member ‘BranchLAN’
set high-availability vrrp sync-group Main member ‘VoIP’
set high-availability vrrp sync-group Main member ‘Network_Appliances’
set interfaces ethernet eth1 address ‘172.16.8.3/24’
set interfaces ethernet eth1 hw-id ‘0c:83:ad:78:5a:01’
set interfaces ethernet eth2 address ‘192.168.8.3/22’
set interfaces ethernet eth2 hw-id ‘0c:83:ad:78:5a:02’
set interfaces ethernet eth3 address ‘10.100.8.6/29’
set interfaces ethernet eth3 firewall local name ‘Prevented_Networks’
set interfaces ethernet eth3 hw-id ‘0c:83:ad:78:5a:03’

20201209 - POC Network

With the Cisco router on and the active VRRP router, pinging from the net_appliance to an IP address in the default VLAN and the phone VLAN, the pings are administratively prohibited by the access-list.

VPCS> sh ip

NAME : VPCS[1]
IP/MASK : 10.100.8.2/29
GATEWAY : 10.100.8.1
DNS :
MAC : 00:50:79:66:68:0b
LPORT : 20250
RHOST:PORT : 127.0.0.1:20251
MTU: : 1500

VPCS> ping 192.168.10.16
*10.100.8.5 icmp_seq=1 ttl=255 time=5.937 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=2 ttl=255 time=7.071 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=3 ttl=255 time=6.357 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=4 ttl=255 time=6.901 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=5 ttl=255 time=7.913 ms (ICMP type:3, code:13, Communication administratively prohibited)

VPCS> ping 172.16.8.1
*10.100.8.5 icmp_seq=1 ttl=255 time=6.050 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=2 ttl=255 time=7.989 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=3 ttl=255 time=7.953 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=4 ttl=255 time=9.409 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.100.8.5 icmp_seq=5 ttl=255 time=8.936 ms (ICMP type:3, code:13, Communication administratively prohibited)

VPCS>

Now, when I turn off the Cisco router, the VyOS router becomes the active one, and a ping is done again as before, and this time the pings are allowed through.

20201209 - POC Network - VyOS Active Router

VPCS> sh ip

NAME : VPCS[1]
IP/MASK : 10.100.8.2/29
GATEWAY : 10.100.8.1
DNS :
MAC : 00:50:79:66:68:14
LPORT : 20266
RHOST:PORT : 127.0.0.1:20267
MTU: : 1500

VPCS> ping 192.168.10.16
84 bytes from 192.168.10.16 icmp_seq=1 ttl=64 time=1.532 ms
84 bytes from 192.168.10.16 icmp_seq=2 ttl=64 time=1.271 ms
84 bytes from 192.168.10.16 icmp_seq=3 ttl=64 time=1.454 ms
84 bytes from 192.168.10.16 icmp_seq=4 ttl=64 time=1.315 ms
84 bytes from 192.168.10.16 icmp_seq=5 ttl=64 time=1.258 ms

VPCS> ping 172.16.8.1
84 bytes from 172.16.8.1 icmp_seq=1 ttl=64 time=2.693 ms
84 bytes from 172.16.8.1 icmp_seq=2 ttl=64 time=1.473 ms
84 bytes from 172.16.8.1 icmp_seq=3 ttl=64 time=1.393 ms
84 bytes from 172.16.8.1 icmp_seq=4 ttl=64 time=1.464 ms
84 bytes from 172.16.8.1 icmp_seq=5 ttl=64 time=1.325 ms

VPCS>

vyos@vyos:~$ sh firewall name Prevented_Networks

Rulesets Information

IPv4 Firewall “Prevented_Networks”:

Active on (eth3,LOCAL)

rule action proto packets bytes

5 reject all 0 0
condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0 /* Prevented_Networks-5 */ reject-
with icmp-port-unreachableLOG enabled

10000 accept all 0 0
condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0 LOG enabled

vyos@vyos:~$

Can someone help me out in understanding why the firewall rules in VyOS are not getting triggered/not used?

Thank you!

jg75996 · December 10, 2020, 7:50pm

As another test, I created a simple setup with just the VyOS router and an endpoint.

20201210 - POC Network - VyOS Only Router

vyos@vyos:~$ sh conf comm
set firewall all-ping 'enable'
set firewall broadcast-ping 'disable'
set firewall config-trap 'disable'
set firewall group network-group Internal_Networks network '192.168.0.0/16'
set firewall group network-group Internal_Networks network '172.16.0.0/16'
set firewall ipv6-receive-redirects 'disable'
set firewall ipv6-src-route 'disable'
set firewall ip-src-route 'disable'
set firewall log-martians 'enable'
set firewall name Prevented_Networks default-action 'accept'
set firewall name Prevented_Networks enable-default-log
set firewall name Prevented_Networks rule 5 action 'reject'
set firewall name Prevented_Networks rule 5 destination group network-group 'Internal_Networks'
set firewall name Prevented_Networks rule 5 log 'enable'
set firewall receive-redirects 'disable'
set firewall send-redirects 'enable'
set firewall source-validation 'disable'
set firewall syn-cookies 'enable'
set firewall twa-hazards-protection 'disable'
set interfaces ethernet eth0 address '192.168.10.16/22'
set interfaces ethernet eth0 hw-id '0c:2d:8d:a7:86:00'
set interfaces ethernet eth1 address '172.16.8.1/23'
set interfaces ethernet eth1 hw-id '0c:2d:8d:a7:86:01'
set interfaces ethernet eth2 address '10.100.8.1/29'
set interfaces ethernet eth2 firewall local name 'Prevented_Networks'
set interfaces ethernet eth2 hw-id '0c:2d:8d:a7:86:02'
set interfaces ethernet eth3 hw-id '0c:2d:8d:a7:86:03'
set interfaces loopback lo
set system config-management commit-revisions '100'
set system console device ttyS0 speed '115200'
set system host-name 'vyos'
set system ntp server 0.pool.ntp.org
set system ntp server 1.pool.ntp.org
set system ntp server 2.pool.ntp.org
set system syslog global facility all level 'info'
set system syslog global facility protocols level 'debug'
vyos@vyos:~$

This setup seems to work as expected and pings to the other networks are being rejected.

PC1> sh ip

NAME : PC1[1]
IP/MASK : 10.100.8.6/29
GATEWAY : 10.100.8.1
DNS :
MAC : 00:50:79:66:68:00
LPORT : 20246
RHOST:PORT : 127.0.0.1:20247
MTU: : 1500

PC1> ping 192.168.10.16
*192.168.10.16 icmp_seq=1 ttl=64 time=8.345 ms (ICMP type:3, code:3, Destination port unreachable)
*192.168.10.16 icmp_seq=2 ttl=64 time=1.299 ms (ICMP type:3, code:3, Destination port unreachable)
*192.168.10.16 icmp_seq=3 ttl=64 time=1.160 ms (ICMP type:3, code:3, Destination port unreachable)
*192.168.10.16 icmp_seq=4 ttl=64 time=0.951 ms (ICMP type:3, code:3, Destination port unreachable)
*192.168.10.16 icmp_seq=5 ttl=64 time=1.009 ms (ICMP type:3, code:3, Destination port unreachable)

PC1> ping 172.16.8.1
*172.16.8.1 icmp_seq=1 ttl=64 time=1.463 ms (ICMP type:3, code:3, Destination port unreachable)
*172.16.8.1 icmp_seq=2 ttl=64 time=0.973 ms (ICMP type:3, code:3, Destination port unreachable)
*172.16.8.1 icmp_seq=3 ttl=64 time=1.208 ms (ICMP type:3, code:3, Destination port unreachable)
*172.16.8.1 icmp_seq=4 ttl=64 time=0.964 ms (ICMP type:3, code:3, Destination port unreachable)
*172.16.8.1 icmp_seq=5 ttl=64 time=0.915 ms (ICMP type:3, code:3, Destination port unreachable)

PC1>

vyos@vyos:~$ sh firewall name Prevented_Networks

Rulesets Information

IPv4 Firewall “Prevented_Networks”:

Active on (eth2,LOCAL)

rule action proto packets bytes

5 reject all 10 840
condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0 /* Prevented_Networks-5 */ reject-
with icmp-port-unreachableLOG enabled

10000 accept all 0 0
condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0 LOG enabled

vyos@vyos:~$

bcstechdept · December 10, 2020, 9:22pm

Hi,
Just a quick observation looking at your config: setting the firewall as “local” under the interfaces will only filter traffic destined to/originating from the VyOS router itself. You’d probably want to use the “in” or “out” keyword depending on your situation.

jg75996 · December 10, 2020, 9:47pm

Hello bcstechdept,

I had those statements in the configuration earlier when testing and was also able to ping and get replies. I did not include those in the original post thinking they were not needed yet.

I have added those statements back into the configuration of the VyOS router:

set interfaces ethernet eth3 address ‘10.100.8.6/29’
set interfaces ethernet eth3 firewall in name ‘Prevented_Networks’
set interfaces ethernet eth3 firewall local name ‘Prevented_Networks’
set interfaces ethernet eth3 firewall out name ‘Prevented_Networks’
set interfaces ethernet eth3 hw-id ‘0c:83:ad:78:5a:03’

Then I did another set of pings using not only the router interfaces, but also the two network endpoints on the respective VLAN1 and VLAN700 networks; 192.168.8.74/22 and 172.16.8.74/24.

VPCS> ping 192.168.10.16
84 bytes from 192.168.10.16 icmp_seq=1 ttl=64 time=1.413 ms
84 bytes from 192.168.10.16 icmp_seq=2 ttl=64 time=1.211 ms
84 bytes from 192.168.10.16 icmp_seq=3 ttl=64 time=1.542 ms
84 bytes from 192.168.10.16 icmp_seq=4 ttl=64 time=1.209 ms
84 bytes from 192.168.10.16 icmp_seq=5 ttl=64 time=1.113 ms

VPCS> ping 192.168.8.74
192.168.8.74 icmp_seq=1 timeout
192.168.8.74 icmp_seq=2 timeout
84 bytes from 192.168.8.74 icmp_seq=3 ttl=63 time=1.928 ms
84 bytes from 192.168.8.74 icmp_seq=4 ttl=63 time=2.070 ms
84 bytes from 192.168.8.74 icmp_seq=5 ttl=63 time=2.022 ms

VPCS> ping 172.16.8.1
84 bytes from 172.16.8.1 icmp_seq=1 ttl=64 time=1.592 ms
84 bytes from 172.16.8.1 icmp_seq=2 ttl=64 time=1.165 ms
84 bytes from 172.16.8.1 icmp_seq=3 ttl=64 time=1.231 ms
84 bytes from 172.16.8.1 icmp_seq=4 ttl=64 time=1.117 ms
84 bytes from 172.16.8.1 icmp_seq=5 ttl=64 time=1.180 ms

VPCS> ping 172.16.8.74
172.16.8.74 icmp_seq=1 timeout
172.16.8.74 icmp_seq=2 timeout
84 bytes from 172.16.8.74 icmp_seq=3 ttl=63 time=2.292 ms
84 bytes from 172.16.8.74 icmp_seq=4 ttl=63 time=1.922 ms
84 bytes from 172.16.8.74 icmp_seq=5 ttl=63 time=2.135 ms

From the VyOS router:
vyos@vyos:~$ show firewall name Prevented_Networks

Rulesets Information

IPv4 Firewall “Prevented_Networks”:

Active on (eth3,IN) (eth3,LOCAL) (eth3,OUT)

rule action proto packets bytes

5 reject all 0 0
condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0 /* Prevented_Networks-5 */ reject-
with icmp-port-unreachableLOG enabled

10000 accept all 10 840
condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0 LOG enabled

vyos@vyos:~$

Thank you!

bcstechdept · December 10, 2020, 10:05pm

Ah, alright. I’ll try and lab this when I get home and see if it will shed some insight. That firewall direction was the only thing I noticed at a quick glance.

helushune · December 11, 2020, 6:08am

The answer probably lies in the Zone-based policy. Here’s the firewall documentation as well. I don’t have any experience with zones on VyOS yet and it’s been so long since my EdgeOS I don’t remember the more advanced things.

There’s a bunch of edits to this post because, “don’t work late, you’ll miss the small things”.

This works for me:

firewall {
    group {
        network-group PROTECTED-RANGES {
            network 172.16.0.0/24
            network 192.168.0.0/24
        }
    }
    name PROTECT-INTERNAL {
        default-action accept
        rule 100 {
            action drop
            destination {
                group {
                    network-group PROTECTED-RANGES
                }
            }
            protocol all
        }
    }
}
interfaces {
    ethernet eth1 {
        address 10.1.0.1/24
        firewall {
            in {
                name PROTECT-INTERNAL
            }
        }
    }
    ethernet eth2 {
        address 172.16.0.1/24
    }
    ethernet eth3 {
        address 192.168.0.1/24
    }
}

ex01

Client 10.1.0.5 cannot reach 172.16.0.5 or 192.16.0.5 but can still reach the gateway. Later, I added an outside route and 10.1.0.5 can reach out as well.

Clients 172.16.0.5 and 192.168.0.5 can see each other:

jg75996 · December 11, 2020, 2:13pm

Hello helushune,

You may be correct about using a Zone-based policy. There was a post I found earlier, titled ACL [SOLVED], that hints at a similar configuration. I will read up on what was linked in your post.

Thank you!