Populating FW groups w/ external IP lists

Hello everyone,
I am in the process of evaluating the migration of a few routers from OPNSense to Vyos. The main reason for this being better/easier support for scripting and ansible. While I am more than happy with Vyos’ routing capabilities; there is one FW feature that is missing (Or I am missing something?)

In OPNSense I have been using “URL(IP)aliases” in FW rules to block inbound access from TOR exit nodes or certain geographical regions. Now, the latter can be achieved with Vyos builtin geoip feature and it works as it should. But what about implementing other blocklists (e.g. Tor exit nodes, DROP lists, etc ?)
A search on the forum suggests the use of external scripts which either

  1. loop through a text file and feeding it line by line to the set firewall command (and therefore spamming the config) or
  2. doing the same but feeding it to nft directly which results in those changes to be lost after reboot or even after making unrelated changes to the firewall.
    Both are not viable options.

So far I have been using the 1.4.2 LTS version - which offers so called “dynamic-groups” but lacks documentation on how to properly populate them from an external URL or even a local text file.
The next step was trying the latest rolling release (dated from 25/6/6) which is supposed to offer remote groups. This feature comes close to what I was looking for but unfortunately it does not populate the group it creates.

For example:
‘set firewall group remote-group address-group country_XX url https://www.ipdeny.com/ipblocks/data/countries/xx.zone

Will not work as intended and adds no IP addresses to the group. The same goes with local files.

Now here are my questions:

  1. What is the current best practice way of achieving the above with 1.4.2 LTS?
  2. How to get the newly implemented function in rolling working?

Thanks in advance

I’m not going to be able to answer your last two questions in any meaningful way, but you are misunderstanding dynamic-groups.

The are described quite clearly here (at least I think it’s pretty clear)

The idea is you can have something happen in a firewall rule, i.e. a probe tries to hit port 22, and the source IP of that probe can be added to a dynamic firewall rule that you apply elsewhere.

  1. Block all packets on the “bad probe” firewall group.
  2. A packet comes in and hits a port 22 rule. The action is to add them to the “bad probe” group
  3. The next packet from “bad probe” that comes in won’t make it to step 2, it’ll get dropped at step 1.

You can apply a timeout so an entry drops out of the group after a set time period, so you might block a badprobe ip for 24 hours etc.

As for the best way to import remote lists into VyOS, as you have noted the groundwork for that exists in 1.5/Rolling, but there is no such feature in 1.4.x. In 1.5 remote-groups should do what you want, but I don’t believe at the moment there’s any way for them to auto-update.

I use a custom-written script to import the spamhaus do-not-route list onto my device every 2-3 days, but it’s pretty clunky so I’m not going to share it (sorry)

You can read about the remote-group feature in 1.5 here. It will attempt to update the list periodically, and fallback to a cached version if the remote file can’t be fetched: Firewall groups — VyOS 1.5.x (circinus) documentation

For 1.4.2, any script you use to update nftables can be put in a post boot/commit script so it’s always updated when you reboot or make changes.

Thanks for the input so far. I do understand that 1.4.x has no built-in option to import remote lists. Regarding 1.5.x (rolling), I am not even able to populate the group once while the group is created so I wasn’t able to test whether it auto updates or not. A quick search has suggested that

firewall global-options resolver-interval 10

is able to achieve this. The issue at this point is that once I created a remote group as mentioned in my first post, the group does not get populated with the IPs/CIDRs which are contained in the remote list. The list can be fetched with curl and shows as one entry per line. So this should not be the issue. I have even tried to run a local webserver and created a file with a few entries manually which also is not read into the remote-group.

Are you using a remote-group or dynamic-group?

In this example I am using a remote-group (on the current rolling release):

set firewall group remote-group address-group country_XX url https://www.ipdeny.com/ipblocks/data/countries/xx.zone

sh firewall group remote-group country_XX
will only show the URL but no IP/CIDR entries.

run sh firewall group country_XX
will show only the URL under “members”

same for run sh firewal group country_XX

sudo nft list ruleset
does not show any elements for the group, unlike groups to which I have IPs manually

You keep showing a config with a dynamic-group configured instead of a remote-group. A dynamic group is different like @tjh stated in his reply.

The above was a typo, the actual command was using ‘remote-group’. I have corrected it in my previous reply.

It’s working fine for me in 2025.06.04-0020-rolling:

set firewall group remote-group test_us url 'https://www.ipdeny.com/ipblocks/data/countries/us.zone'
vyos@vyos# sudo nft get element ip vyos_filter R_test_us { 4.2.2.2 }
table ip vyos_filter {
        set R_test_us {
                type ipv4_addr
                flags interval
                auto-merge
                elements = { 3.0.0.0-4.255.255.255 }
        }
}

Thank you for testing it.
I was not able to reproduce the same result on 2025.06.06-0019-rolling but will try to test this on 2025.06.04-0020-rolling in the next days.

My results are:

set firewall group remote-group test_us url 'https://www.ipdeny.com/ipblocks/data/countries/us.zone'

vyos@vyos-router# sudo nft get element ip vyos_filter R_test_us { 4.2.2.2 }
Error: Could not process rule: No such file or directory
get element ip vyos_filter R_test_us { 4.2.2.2 }
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

May I ask why not all elements are listed inside the brackets?

The get command is looking for an element that matches the IP. So it’s only returning a single element. If you do nft list, it’d return all of them.

I just tested on 2025.06.06-0019-rolling and it worked fine there as well.

Thank you for your clarification regarding the element match.
I have tried with sudo nft list ruleset which returns (amongst others)

 set R_test_us {
                type ipv4_addr
                flags interval
                auto-merge
        }

As you can see, it is lacking the element section which usually shows IPs if added manually.

You may have an issue resolving dns or something. I didn’t test if it returns any failures if it can’t pull down the file.

The problem is not related to DNS:

vyos@vyos-router# dig https://www.ipdeny.com/ipblocks/data/countries/us.zone

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> https://www.ipdeny.com/ipblocks/data/countries/us.zone
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 14820
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;https://www.ipdeny.com/ipblocks/data/countries/us.zone.        IN A

;; AUTHORITY SECTION:
zone.                   1800    IN      SOA     v0n0.nic.zone. hostmaster.donuts.email. 1750224954 7200 900 1209600 3600

;; Query time: 24 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Wed Jun 18 14:52:34 JST 2025
;; MSG SIZE  rcvd: 151

[edit]
vyos@vyos-router#

I will look into this issue tomorrow. Maybe a last question for now:
I would have assumed that the set firewall group remote-group just read in the IPs/CIDRs regardless whether this group is actually used in a firewall rule. Your example suggests that you are actually using a rule called vyos_filter. Could this be the problem?

No, I’m not using it in a rule. The table the set gets created in is called vyos_filter.

I did spin up a new VM with VERSION_ID="2025.06.06-0019-rolling" and only set hostname, timezone as well as giving the system a route to the internet and access to a DNS server. After that I was able to reproduce your result:

vyos@vyos-testing-1# sudo nft get element ip vyos_filter R_test_us { 4.2.2.2 }
table ip vyos_filter {
        set R_test_us {
                type ipv4_addr
                flags interval
                auto-merge
                elements = { 3.0.0.0-4.255.255.255 }
        }
vyos@vyos-testing-1# run sh firewall group test_us detail
Firewall Groups

 Name         | test_us
 Description  |
 Type         | remote_group
 References   | N/D
 Members      | 1.178.0.0/23 1.178.4.0-1.178.31.255 1.178.64.0/23
              | 1.178.68.0-1.178.79.255 1.178.81.0/24 1.178.86.0-1.178.95.255
              | 1.178.100.0-1.178.111.255 1.178.144.0-1.178.174.255
              | 1.178.180.0-1.178.207.255 1.179.2.0/23 1.179.14.0/23
              | 1.179.24.0-1.179.39.255 1.179.52.0-1.179.61.255 1.179.64.0/21
              | 1.179.80.0/21 1.179.100.0-1.179.111.255 2.56.8.0/23 2.56.11.0/24
              | 2.56.32.0/22 2.56.164.0/22 2.57.28.0/22 2.57.68.0/22 2.57.76.0/22
              | 2.57.164.0/22 2.57.232.0/22 2.57.248.0/22 2.58.176.0/22
               ........................truncated...................(but all entries are there!)

Since this thread was about asking how to properly populate FW groups, I will mark your answer as the solution and suggest that this thread can be closed. I will continue to do some testing with this new feature, especially regarding the ability to automatically update this remote-group. Thanks to everyone who gave input on this issue.

Do you have policy based routing in your original config?

Not yet. The goal at this stage was setting up FW groups that can be fed from remote sources and eventually being used in FW rules on the WAN interface (inbound).

Gotcha, policy routes also use nftables and likely would overwrite the elements in the set, which is why I asked.

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.