Any differences in firewall configuration syntax between 1.4-rolling-202312140147 and 1.4.0-epa1?

justinallen03 · February 23, 2024, 12:16am

Hi,

I upgraded my VyOS router according to the versions listed in the title and I have lost DNAT capability. From the firewall (‘show firewall’) it looks like no traffic is reaching my DNAT rule, and the log shows that incoming traffic is dropped by the default rule. It worked fine as configured in 1.4-rolling-202312140147. Any help would be appreciated.
for reference:
Input filter rules:

filter {
default-action drop
enable-default-log
rule 1 {
action jump
jump-target ALLOW-ESTABLISHEDv4
}
rule 2 {
action drop
state invalid
}
rule 3 {
action jump
destination {
port 2222
}
jump-target VyOS_MANAGEMENT
protocol tcp
}
rule 10 {
action accept
connection-status {
nat destination
}
state new
}
rule 20 {
action accept
inbound-interface {
group INTERNAL
}
protocol all
}
rule 30 {
action jump
inbound-interface {
group WAN
}
jump-target ALLOW-PINGv4
}
rule 50 {
action accept
description ALLOW-LOCALHOST-INv4
inbound-interface {
name lo
}
}
rule 100 {
action accept
description ALLOW-DHCP-RENEWv4
destination {
port 68
}
inbound-interface {
group WAN
}
protocol udp
}
rule 120 {
action accept
description ALLOW-IGMP
inbound-interface {
group WAN
}
protocol igmp
}
}

names:
name ALLOW-ESTABLISHEDv4 {
default-action return
rule 1 {
action accept
state established
}
rule 2 {
action accept
state related
}
}
name ALLOW-PINGv4 {
default-action return
rule 1 {
action accept
icmp {
type-name echo-request
}
limit {
rate 1000/second
}
protocol icmp
state new
}
}
name VyOS_MANAGEMENT {
default-action return
rule 15 {
action accept
inbound-interface {
group INTERNAL
}
}
rule 20 {
action drop
inbound-interface {
group WAN
}
recent {
count 4
time minute
}
state new
}
rule 21 {
action accept
inbound-interface {
group WAN
}
state new
}
}

tomcat667 · February 23, 2024, 6:48am

you can fine more information in /config/vyos-migrate.log then you can see which config is migrated. then could be that the syntax is changed.

Edit:

do following

cat  /config/config.boot | vyos-config-to-commands  >> /home/vyos/new.config

you can see differences between the commands

sarthurdev · February 23, 2024, 9:01am

There shouldn’t be any changes since December that would affect your rules. Can you provide output from sudo nft list table ip vyos_filter and sudo nft list table ip vyos_nat

n.fort · February 23, 2024, 10:15am

Destination NAT happens before firewall filters.
So first you need to check if NAT entry is increasing counters, or enable log for desired entry.

justinallen03 · February 23, 2024, 2:53pm

Thanks for your help. Unfortunately I can’t seem to find the issue.

The Destination nat counters are increasing when I try to access that rule remotely. the input rule for DNAT is not getting triggered (firewall counters not increasing)

DNAT stats…(rule 10 is router ssh access)
Rule Packets Bytes Interface

10 1 60 @I_WAN
20 2 120 @I_WAN
30 61 3500 @I_WAN
40 0 0 @I_WAN
200 0 0 dum0

Firewall input filter stats…(rule 10 not processing any packets)
Rule Action Protocol Packets Bytes Conditions

1 jump all 1445 201067 jump NAME_ALLOW-ESTABLISHEDv4
2 drop all 17 684 ct state invalid
3 jump tcp 0 0 tcp dport 2222 jump NAME_VyOS_MANAGEMENT
10 accept all 0 0 ct state new ct status == dnat accept
20 accept all 475 95767 iifname @I_INTERNAL accept
30 jump all 177 20717 iifname @I_WAN jump NAME_ALLOW-PINGv4
50 accept all 6 424 iifname “lo” accept
100 accept udp 0 0 udp dport 68 iifname @I_WAN accept
120 accept igmp 2 72 meta l4proto igmp iifname @I_WAN accept
default drop all 162 19553

justinallen03 · February 23, 2024, 3:05pm

Thanks for helping. Here are the outputs:

table ip vyos_filter {
set RECENT_NAM_VyOS_MANAGEMENT_20 {
type ipv4_addr
size 65535
flags dynamic
}

    set A_LOCALv4 {
            type ipv4_addr
            flags interval
            auto-merge
            elements = { <<>> }
    }

    set A_SERVERS {
            type ipv4_addr
            flags interval
            auto-merge
            elements = { <<>> }
    }

    set I_INTERNAL {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1",
                         "eth1.10",
                         "eth1.20" }
    }

    set I_LAN {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1.10" }
    }

    set I_MEDIA {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1.20" }
    }

    set I_MGMT {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1" }
    }

    set I_WAN {
            type ifname
            flags interval
            auto-merge
            elements = { "eth0" }
    }

    chain VYOS_FORWARD_filter {
            type filter hook forward priority filter; policy accept;
            ct state invalid counter packets 6 bytes 240 drop comment "ipv4-FWD-filter-1"
            iifname @I_INTERNAL counter packets 28643 bytes 4908261 jump NAME_ALLOW-PINGv4 comment "ipv4-FWD-filter-2"
            ct state new ct status == dnat counter packets 0 bytes 0 accept comment "ipv4-FWD-filter-10"
            ip daddr @A_SERVERS iifname @I_INTERNAL counter packets 0 bytes 0 accept comment "ipv4-FWD-filter-50"
            iifname @I_WAN oifname @I_MGMT counter packets 0 bytes 0 jump NAME_WAN-MGMTv4 comment "ipv4-FWD-filter-100"
            iifname @I_WAN oifname @I_LAN counter packets 24100 bytes 6570640 jump NAME_WAN-LANv4 comment "ipv4-FWD-filter-101"
            iifname @I_WAN oifname @I_MEDIA counter packets 0 bytes 0 jump NAME_WAN-MEDIAv4 comment "ipv4-FWD-filter-102"
            iifname @I_MGMT oifname @I_WAN counter packets 108 bytes 6548 jump NAME_MGMT-WANv4 comment "ipv4-FWD-filter-200"
            iifname @I_MGMT oifname @I_LAN counter packets 0 bytes 0 jump NAME_MGMT-LANv4 comment "ipv4-FWD-filter-201"
            iifname @I_LAN oifname @I_WAN counter packets 28517 bytes 4900633 jump NAME_LAN-WANv4 comment "ipv4-FWD-filter-300"
            iifname @I_LAN oifname @I_MGMT counter packets 0 bytes 0 jump NAME_LAN-MGMTv4 comment "ipv4-FWD-filter-301"
            iifname @I_LAN oifname @I_MEDIA counter packets 0 bytes 0 jump NAME_LAN-MEDIAv4 comment "ipv4-FWD-filter-302"
            iifname @I_MEDIA oifname @I_WAN counter packets 0 bytes 0 jump NAME_MEDIA-WANv4 comment "ipv4-FWD-filter-400"
            iifname @I_MEDIA oifname @I_LAN counter packets 0 bytes 0 jump NAME_MEDIA-LANv4 comment "ipv4-FWD-filter-402"
            counter packets 360 bytes 20748 log prefix "[ipv4-FWD-filter-default-D]" drop comment "FWD-filter default-action drop"
    }

    chain VYOS_INPUT_filter {
            type filter hook input priority filter; policy accept;
            counter packets 3486 bytes 451391 jump NAME_ALLOW-ESTABLISHEDv4 comment "ipv4-INP-filter-1"
            ct state invalid counter packets 32 bytes 1304 drop comment "ipv4-INP-filter-2"
            tcp dport <<remote port>> counter packets 0 bytes 0 jump NAME_VyOS_MANAGEMENT comment "ipv4-INP-filter-3"
            ct state new ct status == dnat counter packets 0 bytes 0 accept comment "ipv4-INP-filter-10"
            iifname @I_INTERNAL counter packets 1039 bytes 219087 accept comment "ipv4-INP-filter-20"
            iifname @I_WAN counter packets 451 bytes 50181 jump NAME_ALLOW-PINGv4 comment "ipv4-INP-filter-30"
            iifname "lo" counter packets 12 bytes 923 accept comment "ipv4-INP-filter-50"
            udp dport 68 iifname @I_WAN counter packets 0 bytes 0 accept comment "ipv4-INP-filter-100"
            meta l4proto igmp iifname @I_WAN counter packets 6 bytes 216 accept comment "ipv4-INP-filter-120"
            counter packets 418 bytes 47697 log prefix "[ipv4-INP-filter-default-D]" drop comment "INP-filter default-action drop"
    }

    chain VYOS_OUTPUT_filter {
            type filter hook output priority filter; policy accept;
            counter packets 2194 bytes 351311 jump NAME_ALLOW-ESTABLISHEDv4 comment "ipv4-OUT-filter-1"
            ct state invalid counter packets 0 bytes 0 drop comment "ipv4-OUT-filter-2"
            oifname @I_INTERNAL counter packets 0 bytes 0 accept comment "ipv4-OUT-filter-10"
            oifname "lo" counter packets 12 bytes 923 accept comment "ipv4-OUT-filter-50"
            oifname @I_WAN counter packets 298 bytes 25069 accept comment "ipv4-OUT-filter-100"
            counter packets 0 bytes 0 log prefix "[ipv4-OUT-filter-default-D]" drop comment "OUT-filter default-action drop"
    }

    chain VYOS_FRAG_MARK {
            type filter hook prerouting priority -450; policy accept;
            ip frag-off & 0x3fff != 0x0 meta mark set 0x000ffff1 return
    }

    chain NAME_ALLOW-ESTABLISHEDv4 {
            ct state established counter packets 27181 bytes 7016931 accept comment "ipv4-NAM-ALLOW-ESTABLISHEDv4-1"
            ct state related counter packets 503 bytes 44724 accept comment "ipv4-NAM-ALLOW-ESTABLISHEDv4-2"
            counter packets 2096 bytes 311687 return comment "ALLOW-ESTABLISHEDv4 default-action return"
    }

    chain NAME_ALLOW-PINGv4 {
            ct state new icmp type echo-request limit rate 1000/second burst 5 packets counter packets 45 bytes 3348 accept comment "ipv4-NAM-ALLOW-PINGv4-1"
            counter packets 29049 bytes 4955094 return comment "ALLOW-PINGv4 default-action return"
    }

    chain NAME_LAN-MEDIAv4 {
            counter packets 0 bytes 0 accept comment "LAN-MEDIAv4 default-action accept"
    }

    chain NAME_LAN-MGMTv4 {
            counter packets 0 bytes 0 accept comment "LAN-MGMTv4 default-action accept"
    }

    chain NAME_LAN-WANv4 {
            counter packets 28517 bytes 4900633 accept comment "LAN-WANv4 default-action accept"
    }

    chain NAME_MEDIA-LANv4 {
            counter packets 0 bytes 0 jump NAME_ALLOW-ESTABLISHEDv4 comment "ipv4-NAM-MEDIA-LANv4-10"
            counter packets 0 bytes 0 return comment "MEDIA-LANv4 default-action return"
    }

    chain NAME_MEDIA-WANv4 {
            counter packets 0 bytes 0 accept comment "MEDIA-WANv4 default-action accept"
    }

    chain NAME_MGMT-LANv4 {
            counter packets 0 bytes 0 jump NAME_ALLOW-ESTABLISHEDv4 comment "ipv4-NAM-MGMT-LANv4-10"
            counter packets 0 bytes 0 return comment "MGMT-LANv4 default-action return"
    }

    chain NAME_MGMT-WANv4 {
            counter packets 108 bytes 6548 return comment "MGMT-WANv4 default-action return"
    }

    chain NAME_VyOS_MANAGEMENT {
            iifname @I_INTERNAL counter packets 0 bytes 0 accept comment "ipv4-NAM-VyOS_MANAGEMENT-15"
            ct state new iifname @I_WAN add @RECENT_NAM_VyOS_MANAGEMENT_20 { ip saddr limit rate over 4/minute burst 4 packets } counter packets 0 bytes 0 drop comment "ipv4-NAM-VyOS_MANAGEMENT-20"
            ct state new iifname @I_WAN counter packets 0 bytes 0 accept comment "ipv4-NAM-VyOS_MANAGEMENT-21"
            counter packets 0 bytes 0 return comment "VyOS_MANAGEMENT default-action return"
    }

    chain NAME_WAN-LANv4 {
            counter packets 24100 bytes 6570640 jump NAME_ALLOW-ESTABLISHEDv4 comment "ipv4-NAM-WAN-LANv4-10"
            meta l4proto esp counter packets 0 bytes 0 accept comment "ipv4-NAM-WAN-LANv4-20"
            counter packets 252 bytes 14200 return comment "WAN-LANv4 default-action return"
    }

    chain NAME_WAN-MEDIAv4 {
            counter packets 0 bytes 0 jump NAME_ALLOW-ESTABLISHEDv4 comment "ipv4-NAM-WAN-MEDIAv4-10"
            counter packets 0 bytes 0 return comment "WAN-MEDIAv4 default-action return"
    }

    chain NAME_WAN-MGMTv4 {
            counter packets 0 bytes 0 jump NAME_ALLOW-ESTABLISHEDv4 comment "ipv4-NAM-WAN-MGMTv4-10"
            counter packets 0 bytes 0 return comment "WAN-MGMTv4 default-action return"
    }

}

table ip vyos_nat {
set A_LOCALv4 {
type ipv4_addr
flags interval
auto-merge
elements = {<<router_ip>>}
}

    set A_SERVERS {
            type ipv4_addr
            flags interval
            auto-merge
            elements = { <<svr_ip>> }
    }

    set I_INTERNAL {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1",
                         "eth1.10",
                         "eth1.20" }
    }

    set I_LAN {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1.10" }
    }

    set I_MEDIA {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1.20" }
    }

    set I_MGMT {
            type ifname
            flags interval
            auto-merge
            elements = { "eth1" }
    }

    set I_WAN {
            type ifname
            flags interval
            auto-merge
            elements = { "eth0" }
    }

    chain PREROUTING {
            type nat hook prerouting priority dstnat; policy accept;
            counter packets 5630 bytes 429587 jump VYOS_PRE_DNAT_HOOK
            iifname @I_WAN tcp dport <<remote_port>> counter packets 7 bytes 420 log prefix "[DST-NAT-10]" dnat to <<local ssh addr/port>> comment "DST-NAT-10"
            iifname @I_WAN meta l4proto { tcp, udp } th dport 80 counter packets 3 bytes 160 dnat to 10.22.10.20 comment "DST-NAT-20"
            iifname @I_WAN meta l4proto { tcp, udp } th dport 443 counter packets 282 bytes 16020 dnat to <<svr_addr>> comment "DST-NAT-30"
            iifname @I_WAN tcp dport 22 counter packets 31 bytes 1860 dnat to <<svr_addr>> comment "DST-NAT-40"
            iifname "dum0" meta l4proto { tcp, udp } counter packets 0 bytes 0 dnat ip prefix to ip daddr map { <<notused ip range>> } comment "DST-NAT-200"
    }

    chain POSTROUTING {
            type nat hook postrouting priority srcnat; policy accept;
            counter packets 4542 bytes 332092 jump VYOS_PRE_SNAT_HOOK
            oifname @I_WAN ip saddr <<lan subnet>> counter packets 4165 bytes 300808 masquerade comment "SRC-NAT-100"
            oifname "dum0" ip saddr <<notused subnet>> counter packets 0 bytes 0 masquerade comment "SRC-NAT-200"
    }

    chain VYOS_PRE_DNAT_HOOK {
            return
    }

    chain VYOS_PRE_SNAT_HOOK {
            return
    }

}

n.fort · February 23, 2024, 4:50pm

If you are applying destination NAT, then mos probably connection (after NAT), is directed to a different host rather than the router.
So firewall rules that you should look in that case is ipv4 forward filter, ant not input.

Check diagram in IPv4 Firewall Configuration — VyOS 1.5.x (circinus) documentation

justinallen03 · February 23, 2024, 6:15pm

I do have an external ssh port forwarded to the router local address port 22, which is why the DNAT rule exists on the input table. The forwarding table also has the DNAT rule for forwarding to other machines on the network.

I did verify that, on the older version, if I disable the DNAT rule in the input filter then I can’t ssh into my router externally.

Looking at the nft output, the only difference I saw between the two was in the DNAT rule syntax…

The old syntax (both input and forward chains):
ct state new ct status dnat counter packets 0 bytes 0 accept comment “ipv4-INP-filter-10”
ct state new ct status dnat counter packets 2 bytes 104 accept comment “ipv4-FWD-filter-10”

the new syntax (both input and forward chains):
ct state new ct status == dnat counter packets 0 bytes 0 accept comment “ipv4-INP-filter-10”
ct state new ct status == dnat counter packets 0 bytes 0 accept comment “ipv4-FWD-filter-10”

I’m not sure if the added “==” changes anything, it was just something I noticed.

cnrd · February 23, 2024, 7:32pm

Hi

I just want to add that I am also seeing issues with DNAT after moving from 1.4.x-rolling-202312290303 to 1.4.0-epa1

Some testing seem to indicate that

     connection-status {
         nat destination
     }

is not working correctly.

If I add explicit rules to the firewall instead, then the traffic starts passing correctly.

In the example below rule 10 and 20 are the old rules while 30 and 40 are new rules that covers part of my DNAT.

 default-action return
 rule 10 {
     action accept
     connection-status {
         nat destination
     }
     source {
         geoip {
             country-code xxx
         }
     }
     state new
 }
 rule 20 {
     action accept
     connection-status {
         nat destination
     }
     source {
         group {
             address-group XXXX
         }
     }
     state new
 }
 rule 30 {
     action accept
     destination {
         address xxx.xxx.xxx.xxx
         port 80
     }
     protocol tcp_udp
     state new
 }
 rule 40 {
     action accept
     destination {
         address xxx.xxx.xxx.xxx
         port 443
     }
     protocol tcp_udp
     state new
 }

azhagan · February 24, 2024, 4:32am

I faced the same issue with connection-status not working, being explicit also worked for me.