My firewalls have quite a bit of RAM and can take advantage of that with ZFS. Is this on the roadmap? I could set it up manually, yet it’d be too much of a pain to maintain on the rolling-release. Has this been evaluated as a possibility for a future release? I’m not sure of the implications withe squash, yet I assume it’s doable.
Thanks for the consideration.
Keith
Nothing like that yet
how will you leverage ZFS on the router?
To store container images? something else?
Yes, and to give a general boost to FS operations. I run ZFS on all of my boxes, and even on those with full flash storage, I see benefits in IO and latency. With regard to the containers, it’d add a potentially sizable boost as well. Other benefits include out-of-the-box volume encryption, compression, CoW, snapshots, etc. There’s a whole ton to gain, and although EXT4 works well, there are advantages to ZFS that aren’t available with EXT4 and would also work well on a router/firewall device, even without the use of containers.
I would say that running ZFS on routers or firewalls is a very bad idea. It’s slower than any filesystem out there. Yes you get the advantage of compression, block checksumming, raidz(1,2,3) or deduplication (don’t use), but do you need it on a more or less static device.
We are running petabytes of ZFS and I’m a very big fan. But using it in routers or firewalls is just generating a lot of overhead unless you got CPU and RAM to spare.
IMHO if you need ZFS for containers on VyOS then setup a dedicated TrueNAS or Unraid box to which you let your containers use ISCSI or whatever to access their ZFS based storage. You could also use Linstor and other storage solutions.
Or if you want to run containers and VM-guests at scale then install Proxmox (or XCP-NG) on dedicated box(es). You can run VyOS as a VM-guest aswell.
From VyOS point of view there is little to no reasons to start using ZFS since the ISO is read during boot into ramdisk and the only thing written by VyOS itself is the few times you commit and save a new config and logfiles.
To me the container support is a “nice to have” so you can run Adguard and whatelse if needed but if you start to use your VyOS installation as some VM-host then you are doing it wrong (and should switch to Proxmox or XCP-NG to be runned on baremetal and run VyOS as VM-guest unless you give the VyOS its own hardware).
In my case, I have plenty of RAM and a processor. I disagree that it’s slower than any filesystem; for me, it’s the fastest I’ve found for most of my operations. I’m not sure where you’d get that from. It does not need to be static; I run it on hypervisors, file servers, etc. It definitely doesn’t have to be a forced install, yet it would be nice to have an option available during installation for those who have available resources. My box is less router and more firewall/security device, so it’s not a bad match. For a PI/Celeron running it strictly as a router, I see no need; otherwise, it may be useful.
So this is actually a good point. As I mentioned, I don’t know a ton about squash, if we’re already running in RAM, there’s no need for ZFS. Is this also true for moving parts, such as logs, or only the OS itself?
Thanks for the education,
Keith
When booting the squashfs is loaded into a ramdisk.
This is then used as the underlay with overflayfs onto it making any changes to this readonly squashfs be saved in the persistent directory for this installation (each installation have its own persistence dir).
This gives that its mainly the configfile and logs who ends up in the persistent directory on your drives and gets written to over time.
That is when you enter lets say /etc and “ls -la” it is the content of the squashfs (now placed in RAM) thats being displayed.
If you change /etc/fstab the changed file will end up at (for example):
/usr/lib/live/mount/persistence/boot/1.5-rolling-202404250024/rw/etc/fstab
So basically what overlayfs does is that it will first look in /usr/lib/live/mount/persistence/boot/1.5-rolling-202404250024/ and use the file found there. If its not there it will use the underlay found in what the squashfs provides as “/”.
This gives that VyOS have very little use of all the features that ZFS provides.
But sure, please feel free to file this as a feature request over at https://vyos.dev
Except for kernel support also userland tools must be included (which will increase the imagesize) and initramfs must include support to be able to fsck during boot if needed.