However, for doing so I need some Details to VyOS Podman integration.
I see 3 attempts:
Use full virtualization to simply get kubelet VMs on top of VyOS which could be controlled from VyOS config utilizing k8s API.
(+) It is the least invasive integration
(+) Would be an intermediate step to create a point from which a direct integration could start
(-) Costs most resources
(-) Requires kernel level work what truly would introduce certain weakness as argued by Apachez
Use privileged container to host kubelet
(+) On first glance similar complexity as 1
(+) Would be an intermediate step to create a point from which a direct integration could start
(-) thinking about netfilter and kernel namespaces I see a lot of corners and edges where such an attempt might simply fail
Deep integration of kubernetes into VyOS using the integrated Podman as driver
(+) Most neat integration
(+) Lowest costs
(-) Huge way to go, for an OSS project maybe not feasible to plan such huge steps.
So the questions:
I do not see how to create VM or privileged pods from documentation
Is KVM even avail- and useable? Oh no!
root@gate:~# lsmod | grep kvm
root@gate:~# modprobe kvm_intel
modprobe: FATAL: Module kvm_intel not found in directory /lib/modules/6.1.43-amd64-vyos
Is anything differing from a full podman you install on debian
Is it planned to extend podman integration with full and privileged virtualization?
Please share your insights, proposals and opinions
Personally even if its a nice a addition and should be doable (afterall - VyOS is a Debian stable (currently bookworm version 12.1) with a nice frontend) VyOS to me is still a router and focus should be to mainly deal with that router stuff before adding more features such as Kubernetes etc (which also means that the attack surface will extend, the main image will grow etc).
If I would to be run virtualization I would install proxmox or xcp-ng or something from scratch such as alpine or debian/ubuntu/devuan on baremetal. This way you also have total control of the box without a frontend in between.
There is a long term goal. If you think this together with ⚓ T5390 VyOS public/private Swarm you might see, why it gets valueable to have a vyos version (not necessarily the “PLAIN” version) which allows to build up user owned servicing swarms. Kind of an edge WWW which allows you to deploy common services together with others rather than renting some datacenter devices to service into 0.0.0.0/0. People could buy https://www.zotac.com/de/product/mini_pcs/zbox-edge-mi646-barebone-0 and join common efforts to build up services within a global “Peoples” Network.
Yes, these are all nice solutions if you think the classic way. The catch about kubernetes is, it is the last evolution of deterministic service setup. But k8s installation itself is “classic” and cumbersome. For standardizing setups you rely on tools like ansible what usually packs complexity on top rather than reducing it. When you want to empower “simple” tech oriented people to actually participate in servicing what is important to them… such systems are useless outside of a datacenter group keeping the same standards sharing the same templates. But when you think about such a thing, you really do not want to push monopoles and especially not datacenter provider monopoles.
As far as I see, VyOS is not meant to be just a router but to redefine how network and service topologies are thought and built. There fore I think it is pretty much on the roadmap, even I would do it when it wasn’s since I simply require it and VyOS seems to be the best platform to build on top.
I asked for and appreciate your opinion. Conclusions from Argument:
“do not mess up hardening simplicity of routing system”
Extension or separate version like “VyOS::Swarm”
I think overlays would be better to have “VyOS::Kube” and “VyOS::Swarm” so that system designers could build up custom system.
Personally I would prefer not to have different images but rather be able to disable or for that matter uninstall certain features.
That is either have a base image where one can append modules/packages (or whatever one want to call it) to it or the other way around have a larger base image but where lets say kubernetes and some more (or all?) are a set of modules/packages (think appimage/flatpak or similar) that through configuration can be removed (to make the install smaller) or at least be able to completely disable stuff that isnt enabled (since having them available do increase the attacksurface).
Drawback with the later is that the size of the image will increase from todays 444 MB.
In comparision Aristas EOS images are today about 1Gbyte of size (and increasing aswell) but thats mainly because they use a single image for ALL of their models (which is a VERY nice feature).
Not that size matters but I remember the days of FloppyFW where linux kernel + iptables + drivers + busybox with its tools + SSH dropbear + QoS (and a tc-script) could fit in a single 1.44 (well actually 1.68) MB floppy and booted blazing fast (if you installed it on a harddrive).
Please see how I updated my answer to your 0,05€
VyOS Images are already OvelayFS so it would just be naturally to put a Kube, Swarm, Podman and whatsoever overlay on top.
overlay on / type overlay (rw,noatime,lowerdir=/live/rootfs/1.4-rolling-202308050917.squashfs/,upperdir=/live/persistence/boot/1.4-rolling-202308050917/rw,workdir=/live/persistence/boot/1.4-rolling-202308050917/work)
I just wonder how much work it would be to rip vyos as it is apart, get a minimal base system and put the other stuff together as overlays.
but I remember the days of FloppyFW where linux kernel + iptables + drivers + busybox with its tools + SSH dropbear + QoS (and a tc-script) could fit in a single 1.44 (well actually 1.68) MB floppy and booted blazing fast (if you installed it on a harddrive).