I am currently setting up my firewall to migrate from OPNSense to VyOS, because my internet speeds are going to increase to 8Gbps and VyOS is faster. It’s really great what you guys are doing and the progress that is being made.
To increase the forwarding speed flowtables is great. And I am glad that it got backported to 1.4. But what is the correct way to implement it? This is my current configuration and the configuration that I have for flowtables with VyOS 1.4-rolling-202311301716:
set firewall flowtable FLOW-DMZ interface ‘eth2’
set firewall flowtable FLOW-DMZ offload ‘software’
set firewall flowtable FLOW-GUEST interface ‘eth3’
set firewall flowtable FLOW-GUEST offload ‘software’
set firewall flowtable FLOW-IOT interface ‘eth4’
set firewall flowtable FLOW-IOT offload ‘software’
set firewall flowtable FLOW-LAN interface ‘eth1’
set firewall flowtable FLOW-LAN offload ‘software’
set firewall flowtable FLOW-WAN interface ‘eth0’
set firewall flowtable FLOW-WAN offload ‘software’
set firewall flowtable FLOW-WIREGUARD interface ‘wg01’
I want to use flowtables on all my interfaces, but how can I check if it’s enabled? Since the command set system conntrack flow-accounting is not working on 1.4.
And is it enough to enable it on all my interfaces, or is it needed to enable it in the firewall rules as well? Because currently I don’t see the option to use flowtables with zone-based firewall
On further investigation it looks like when you enable the forward filter, an implicit accept is added to the firewall. I can’t add a drop rule to the forwarding chain after the offload, because things will brake. The rules on the zone-based firewall will not get hit anymore, because of the implicit accept.
set firewall ipv4 forward filter default-action drop
That would set it globally but you may be better off creating the offload rule within each custom chain you made for zones instead of the forward filter itself.
I used that command, but my zone firewalls rules did not work anymore after that (all that relate to forwarding) The best option would be like you said a offload rule in chain, but currently that option it not available on the CLI, or just global setting to enable flowtables.
Looks like according to ⚓ T4502 Consider implementing (NAT/other) flow table offload it is still a WIP. I don’t know enough of the nitty griity of nftables but I don’t see why offload actions couldn’t be applied to custom chains. Since zones were originally going to be dropped entirely until the reversal of that decision, it’s not unreasonable to presume it was missed as an option. I’d suggest creating an account at https://vyos.dev and updating that task regarding functionality missing from custom chains.
What about defining offfload in custom chain, for example:
# if used WAN_to_DMZ in firewall for traffic that foes from zone WAN to zone DMZ
set firewall ipv4 name WAN_to_DMZ rule XX action offload
set firewall ipv4 name WAN_to_DMZ rule XX offload target FLOW-XXX
Most probably, you will need to offload on the other way round too ( DMZ_to_WAN)
@deallerbeste and myself can both confirm offload isn’t a valid action in custom chains.
[edit]
vy1@vyos# edit firewall ipv4 name CONN_FILTER rule 10
[edit firewall ipv4 name CONN_FILTER rule 10]
vy1@vyos# set action
Possible completions:
accept Accept matching entries
continue Continue parsing next rule
jump Jump to another chain
reject Reject matching entries
return Return from the current chain and continue at the next rule of the last chain
drop Drop matching entries
queue Enqueue packet to userspace
synproxy Synproxy connections
[edit firewall ipv4 name CONN_FILTER rule 10]
vy1@vyos# set action offload
Invalid value
Value validation failed
Set failed
In the meantime. I have a NIC that support hardware offloads (eth0) but it’s not working in combination with software offload. Any ideas what I am doing wrong this time? Eveytime I get the following error message, but I already added this to the configuration:
#Hardware:
set interfaces ethernet eth0 offload hw-tc-offload
set firewall flowtable FLOW-HW interface 'eth0'
set firewall flowtable FLOW-HW offload 'hardware'
#Software
set firewall flowtable FLOW-SW interface 'eth2'
set firewall flowtable FLOW-SW interface 'eth3'
set firewall flowtable FLOW-SW interface 'eth4'
set firewall flowtable FLOW-SW offload 'software'
When I enable this with ethtoool and commit after that I get the following error:
sudo ethtool -K eth0 hw-tc-offload on
Failed to apply firewall: /run/nftables.conf:28:15-36: Error: Could not
process rule: Operation not supported flowtable VYOS_FLOWTABLE_FLOW-
HW { ^^^^^^^^^^^^^^^^^^^^^^ /run/nftables.conf:280:15-36:
Error: Could not process rule: Operation not supported flowtable
VYOS_FLOWTABLE_FLOW-HW { ^^^^^^^^^^^^^^^^^^^^^^
Does it work if you remove the software flowtable? (like only set up hardware for eth0)
If not, it sounds more like your NIC doesn’t support hw flowtables.
Does it work if you only define one flowtable using hardware offload?
I wasn’t able to make it run using hardware offload in my lab. Not even using direct nft commands, even though all requirements are supposed to be satisfied.
