I would like to write a tiny shell for a Linux system I am running.
Since I do not have any experience in this field I thought that the developers of VyOS might give me a good advice on where to start.
Hi @elico , I’m not really a developer, but from what you’re looking for, I think you can take a look to the “argparse” python module which is pretty simple in usage.
argparse doc - argparse — Parser for command-line options, arguments and sub-commands — Python 3.11.3 documentation
usage example from pyneng course - argparse - Python for network engineers
Not required a VyOS developer but someone that understands the concept.
What I want to do is an interactive shell rather a simple CLI tool.
I have seen clish but since it requires XML I believe it’s not the right tool for me.
What I need is something like what VyOS and other embedded devices like EdgeOS and JunOS has.
An interactive shell that I can set values with and then to apply them later on to a DB.
I want to be able to set values and also if possible evaluate some of the definitions.
I have seen that bash completion is pretty nice but I am not sure what would be the right tool for me.
Hi @elico! This is not a “mini/tiny shell” in general, but I can tell you a bit about the inner workings of vyos… 
Vyos’es main cli is based around a bash fork named vyatta-bash or vbash, This is an old fork of bash 4.1 that atm. is over 10years old. This is a normal bash that also implements a few special commands (eg. show…) and creates the “configuration” sub-mode that is lacking from a standard bash, on top of this special bash there is created a configuration system, called vyatta-cfg. It implements the the set/delete/commit++ special commands inside this sub-mode.
The full CLI is composed of multiple templates that describe whats happening when you configure or change a setting… you could look at some of those here: templates from crux
In its basic form the cli only supports running bash and perl scripts… But as this mode is quite limited and its not as easy to develop for it, the vyos team have extended it to also support loading python scripts by the use of the vyos1x-config and libvyosconfig libraries. All scripts are now beeing rerwritten to this new python style, and its created a way to autogenerate the templates.
This is now located inside the vyos-1x repository This code takes a set of “interface-definitions” and generates the correct node.def files for vyatta-bash/vyatta-cfg to use. it also points to the corresponding “executor script” so the correct scripts are executed in the correct order when something is configured inside the cli.
While it should not be “to hard” to use this code for creating a custom cli, it does need a lot of insight into how the code is laid out. and as noted earlier the bash code is quite old, and it does not support some of the new style bash completions that are a part of newer distributions.
As for where the vyos project are headed this code is beginning to enter its legacy state as it is planned to be switched out for a new configuration engine from vyos version 2.0… to look at this work look into the vyconf project…
This is a simplified view of the current cli engine and it might be that i’ve missed on a few details, but i hope this could give you some insight for where to start in your own project
Thanks for the detailed response.
I will try to look at the references you mentioned later on.
From my experience with cli shells until now(which is 6+) out of my sleeve both JunOS and PaloAlto are the most admin friendly to some degree when you look at your device as a piece of Engineering.
While Cisco and couple others have another way to work on things and these are good but at-least to my experience you need to know your stuff around these pretty well to prevent your self from locking out.
One thing that is worth mentioning that I do not know about VyOS is a failsafe for admin configuration/programming errors.
Mikrotik has a very nice feature which is based on a “transaction”, you turn on a “Safe mode” and if for any reason you are kicked out of the system while you didn’t turned off the “Safe mode” then all the changes are being reverted back before the “Safe mode” was turned on.
Mikrotik also has “rsc” scripts which is great and allows many things to happen but this is since they have their own kernel and limited environment.
I have seen in the last month a case which the vendor(which is not mentioned in this post) official tech support was working on a specific setup and caused the device to lock itself and the only option to make the device work again was to reset the device into it’s defaults and then to load the config backup file.
I must admit that this case gave me a whole new perspective on the subject and… since there was a VyOS 2 and other things on the table I must admit that it’s pretty important to look at the list of things which others offer in their systems/cli/web/other and try to base something in VyOS from these.
I still believe that a web interface is a good thing but there is a cost for that and I am not sure but do believe that a good enough CLI foundation can help to build the whole system differently and based on that allow the system to be built to another level.
I would say that it would be very nice to have a very good looking and working WEB and or other interfaces but lesson learned from other vendors like Juniper, PaloAlto and Cisco is that it would require CPU, RAM and DISK to allow all these to be good.
The lesson I took from Mikrotik about many things is that it’s possible to build something amazing with little resources.
(VyOS is amazing if you take into account a few little things)
I will try to respond after I will review some of the references.
And again,
Thanks!
if you ask me, the most important features when designing user friendly cli’s are to have some form of two-stage commit process, so the user is able to review its changes before it is activated on the device… these features should also verify as much as possible of the user-input to be able to catch issues before it is allowed to be commited into the system, but more on that later
These kind of features are called “Atomic Updates”, where the device will automatically rollback the configuration in case of an exception and/or no acknowledgement from the user after the configuration is commit’ed. This is for now only partially supported in vyos because of the vyatta-bash/vyatta-cfg framework it is using. it is also quite hard to get working in a effective manner, and will be a rather important part of the new vyconf framework when its completed.
For now in vyos this works in a “feature by feature” base, as the new configuration scripts does a lot of checks before it starts configuring its backend processes… but vyatta-cfg does only do one “pass” for each configuration script before it continues to the next… each script is then executed in a prioritised order and code is “commited” feature by feature until something breaks… then the script execution is terminated at the point of an error, and the failing feature and other things still not commited are left uncommited until the user fixes these faults and the engine can continue.
Because of the large amount of internal tests done before a feature is commited we hope to catch MOST of the errors that can happend and lead to a backend exception… but there is no guarantee for this…
This is also a issue for vyos thats needed to be solved by vyconf, the backend configuration daemon needs to be able to take in “candidate” configuration from more than one type of source… the current engine only does cli, and all web/api++ access needs to emulate a cli session to be able to use the configuration sub-system… and this can be a big issue and is kinda “hackish” when doing anything other than cli…
1 Like
Just to mention the goal is: to create a cli shell like vtysh of frr for Squid-Cache.
The setup is based on couple external helpers and a DB so the updates eventually are kind of “atomic”.
I will try to use the two stage commit to allow “compare” ie “diff” and also testing of the new configuration to some degree.
Currently I did one nice thing with network namespaces and virtual routers to allow the creation of atomic updates for nftables rules at the next load balancer project:
it’s possible to run a similar idea with a docker/podman container on Squid-Cache configuration testing.
There are couple ways to implement the same network environment, ie synthetic or real.
For components like nftables or iptables you can use dummy network interfaces inside a namespace however for other components there is a need for other measures to verify the configurations.
I would probably need to create the proxy application “open” library which is based on couple concepts of ntop ndpi and squid acls.