I’m trying to understand the internal architecture of the Vyatta, specially so I understand what I can and can’t do in terms of adding new packages on a running machine.
I understand that underneath it uses Debian and makes use of Unionfs, but I trying to understand how the unionfs is used (I assume to have clean conig copies), what is persistent between reboots, what is RO and what is not, what gets wiped during a reboot and so forth.
I’m hoping somebody has a Vyatta Internal manual/presentation!
I’m an experienced Linux admin and manage 48 Vyattas at this point, so I’m really looking for the hardcore stuff 8-)!
It’s really just debian repos. The problem you’ll have is every image-upgrade (the only supported update method) will wipe out any additional deb’s you’ve installed. In terms of persistence, I think everything under /config and a few select other files (copied during the ‘add system image’ process).
The approach I’ve been taking is building custom deb’s, installing them (manually or via a self-hosted repo) and then if I’m happy with something, I’ll bake an ISO against latest release with my changes.
The changes I’ve done so far have been relatively minor though, although I’m wanting to extend with some other open source tools.
I’m interested to know what packages you are planning to add too if you are willing to share?
Appreciate the answer. I’m planning on installing snort, so I have a IDS inspecting the traffic as it flows though the Vyatta.
Do you have more information about the “add system image” process? I initially thought that basically that would add a new kernel, but I realized that it must also update some other system binaries, but now I’m not sure anymore.
Snort can be installed from debian squeeze repos although it’s v22.214.171.124-8 which appears to be old. I’ve been playing with openappid with much newer versions of snort but haven’t tried to make it work on VyOS/Debian Squeeze yet
For add system image, I not intimately familiar with the mechanism (a quick look at the perl code didn’t make it easy to follow the logic), but it ultimately installs a squashfs file, kernel & initrd under /live/image/boot/
As I alluded to, there is some logic in there to copy/persist SSH keys etc, and anything under /config/ appears to persist also.
If you wanted to go to some effort of ensuring changes are applied between upgrades, but don’t want to build an image yourself, then /etc/rc.local (non-persistent) sources /opt/vyatta/etc/config/scripts/vyatta-postconfig-bootup.script, which is persistent, so you can put any magic in there (e.g 'apt-get -y snort; cp /config/snort/snort.conf /etc/snort/snort.conf, or whatever).
A wildly different idea I had with VyOS was to get docker support in the main image (someone has already done some early work/testing and published in the forums), and then things like IDS/very new versions of snort etc can be ran in docker containers while passing through access to host resources (e.g. host networking interfaces to listen on). That would help people wanting to do bleeding edge stuff with their reliable VyOS devices, while minimally touching the VyOS filesystem. Haven’t explored this beyond a few passing thoughts though