VyOS 1.2.0 build for 586?

njh · January 19, 2019, 1:39pm

Hi,

I hoping to use VyOS for my home router. I have an PC Engines ALIX 3C board with an AMD Geode processor on it - 586 architecture. I have successfully installed VyOS 1.1.8 on it.

But given that I am new to VyOS, it would make sense to start with 1.2.0 but there doesn’t seem to be a 586 build of it? Will that be coming in the future?

Thanks,

nick.

dmbaturin · January 19, 2019, 3:56pm

Hi Nicholas,
There is nothing that prevents one from making a 32-bit build technically. All it takes is to recompile the platform-dependent packages and rebuild the image.

The demand for it is very low and shrinking though, now that affordable 64-bit boxes are common. The question therefore is whether we can make 32-bit builds easy enough to create a low maintenance build setup.

So far live-build couldn’t make images for architectures other than the on its running on. If someone finds a way to do that, it may tip the balance. Else we’d need to maintain a 32-bit host just to make those images.

njh · January 19, 2019, 8:05pm

Hi,

Disappointing but I can understand your position.

I would have thought that compiling for 586 should be easier than for ARM, as you don’t need a separate toolchain and 586 code runs on amd64 but I guess it is still work and more complex when there is a lot of different modules / dependencies.

nick.

dmbaturin · January 19, 2019, 8:38pm

You do need a separate toolchain for 32-bit x86 code. Building ARM binaries on an ARM host is just as easy as building amd64 binaries on amd64. Building i586 binaries on an amd64 host is just a little easier than building them on an ARM host, it still needs a separate toolchain and libraries, even though testing those binaries is much easier because you can run them. You could simplify it by running builds in a containers in the i586/amd64 case, but the procedures still need to be developed.
You also need a separate build host for images because live-build’s architecture option appears to have no effect, at least when you specify i586 for live-build running on an amd64 host.

We don’t get many requests for i586, and none of the maintainers or active contributors use it (likewise, no one offered to pay for it), so other issues naturally get higher priorities. None of us is categorically opposed to i586 images though, so if someone helps make it happen, it may happen.

njh · January 20, 2019, 4:03pm

I have had a go at using the vyos-build repo to create a VyOS 1.20 build for i386 / i586.

Docker doesn’t seem to be able to use a 32-bit image, while running on a 64-bit host, so I looked at using a virtual machine instead.

I created a 32-bit VirtualBox machine and installed Debian Jessie using the 32-bit netinstall ISO
I installed the build dependency packages listed in the Dockerfile by running the RUN commands manually
The command that downloads and installs packer was hard-coded to use amd64 - but there is also a 386 version available
I ran ./configure --build-type development --architecture i586 to configure the build
I used the ./scripts/build-submodules to try and build the Debian package files
I then ran make iso to try and build an ISO

I was able to build a large number of the VyOS packages without modification. The ones that failed to build didn’t seem to relate to the architecture - I am not sure why they are in a broken state?

Missing external package dependencies:

libdigest-sha1-perl: should this be upgraded to libdigest-sha-perl?
libxtables10 : depended on by iproute2 - not sure why it isn’t being found in jessie: Debian -- Error
libssl1.0.0: depended on by keepalived (not sure why it isn’t being picked up from jessie-backports)
libradiusclient-ng2: not sure where this is supposed to come from?
libboost-filesystem1.42-dev: jessie has libboost-filesystem1.55-dev - can/should it be upgraded?

Unable to build:

vyatta-wanloadbalance: Compile error: src/main.cc:73:47: error: ‘getopt’ was not declared in this scope
vyatta-webgui: Compile error: fatal error: glibconfig.h: No such file or directory
vyatta-ipv6-rtradv : dh_install: vyatta-ipv6-rtradv missing files: opt/vyatta/etc

I don’t know how to work out how to build - not listed in the build-submodules script:

vyatta-busybox
vyatta-dhcp3-client
vyatta-dhcp3
vyatta-ipsec
vyatta-quagga
vyatta-unicast
vyos-1x

This is a massive rathole for me! But if I get the same problems trying to build using standard docker build for amd64, should I report them as bugs?

njh · January 21, 2019, 10:05am

After noticing that vyatta-busybox shouldn’t be used at all - I just realised that I was using the ‘current’ branch and not ‘master’

So I will start again will all this…

njh · January 21, 2019, 11:18pm

I had a lot more success getting the current branch to build
All the main packages built fine without any problems.

Getting vyconf and libvyosconfig and was a bit of work - but it looks like there have been some improvements to the build process today
Getting FRR to build was a little painful but I got there in the end
The biggest task is going to be to create a kernel configuration and get it to build
I think vyos-wireguard and vyos-accel-ppp packages depend on the kernel

So I will try and get a kernel build to work next… hopefully I’m not too far off having a working ISO.

njh · January 25, 2019, 10:37am

I think it might be time for me to give up trying to create a build for 586, after spending far too much of my time on it over the past 7 days.

Building all the dependencies (some not even listed in the project) took a very long time after resolving multiple problems
I eventually got to the point where I had an ISO: vyos-1.2.0-rolling+201901242343-i386.iso, however when I booted it, I was unable to login to it. Upon inspection of the chroot, it looks like no vies:vyos user was created. With no user with any password, I was unable to login.
During the boot sequence a service failed to start because libyang was missing.

These leads me to question if I was even building from the correct branch/version of vyos-build.

I really wanted to get VyOS up and running but I sadly I think I have spent too much time on it and it is time went back to using raw Debian Linux + ansible.

dmbaturin · January 25, 2019, 1:29pm

Hi @njh,
Yes, that’s pretty much why there’s no i586 image yet. There’s lots of things to clean up before it can be done easily. That work is ongoing, but it takes time, a lot of it.

Which branch were you building from? For 1.2.0/future LTS you need “crux”, and for the most recent you need “current”.

Thanks for your input though. Now to the details:

vyatta-quagga: no longer used, we use FRR now. I need to remove all references to it.
vyatta-busybox: I think it’s no longer used, but we need to check carefully.
vyatta-unicast: it was a virtual package that could be provided by either Quagga or proprietary ZebOS
libradiusclient-ng2: this is @Unicron’s work, I’ll check with him.
vyatta-webgui: this is about to be phased out.
vyatta-ipv6-rtradv: I think it’s no longer needed.
vyatta-wanloadbalance: this is odd, it builds for me on amd64. I need to check.
libdigest-sha1-perl: I wonder what depends on it, but yes, we should upgrade if needed.
libboost-filesystem1.42-dev: we made it alternate dependency in some but I think not all packages.
Wireguard and accel-ppp indeed depend on the kernel, I have the procedure documented in Build VyOS — VyOS 1.4.x (sagitta) documentation

njh · January 25, 2019, 10:31pm

Thanks for the reply.

A lot of my initial problems were because I was building from master and not current (or crux) - most of the issues you covered above were resolved by switching to the current branch.

I am not sure if I have a complete build or not now, but the problem I have is with there being no vyos user on the live build. Do you know which script/component is supposed to create that user and set the password to ‘vyos’?

hagbard · January 25, 2019, 11:45pm

AFAIK, is set via /config/config.boot which triggers a script when the config is applied and creates the user when it dosn’t exist. (see: /opt/vyatta/share/vyatta-cfg/templates/system/login…)

njh · January 26, 2019, 12:30am

@hagbard thanks! That is really helpful. I assumed that the live ISO build process would have the user pre-installed in in /etc/passwd and /etc/shadow.

I have an error during startup saying “error while loading shared libraries: libyang.so.0.16”

So it is possible that the process that is supposed to be configuring VyOS is failing and the user isn’t being created.

hagbard · January 26, 2019, 4:59pm

wireguard and accel-ppp needs the kernel sources. Wireguard could run purely in userspace, but it sucks performance of the cpu. We usually have that in the RERADME.md (GitHub - vyos/vyos-accel-ppp: accel-ppp debianized for the use with vyos, GitHub - vyos-legacy/vyos-wireguard). I surely have to make mine more structured and readable like the one from wireguard :), but it’s all compiling. I never have tested it against a x86 only, but did a cross-compile for armhf with no issues. For your user, I think it’s less a question of the ‘process’, at the end of the boot sequence you have a vyos-router and config, if anything exits 1, you would see that as error.
What you can do if you can boot is, booting in single user and init=/bin/bash, that way you are root on the system (there are ways to prevent that switch which might be activated in later releases) and can investigate. Wiorst case start vyos-config and trigger the user creation manually, or set everything debug and reboot etc.
Just a few ideas.
Hope that helps.

njh · January 27, 2019, 5:59pm

I have got a little bit further this weekend.

I resolved the problem with libyang thanks to this commit by c-po:

I put the resulting Debian package into the packages directory and it successfully included it in the build (previous attempts to install the libyang package in live-build failed for various annoying/unknown reasons)

Following @hagbard’s advice I managed to login to the VyOS Live CD by:

Pressing tab in the isolinux boot screen to edit the Linux kernel parameters
Appending single to the end of the parameters
When the boot paused I used passwd to set a root password and then pressed control-d to continue the boot
I could then login as root. However show configuration showed that there was no configuration loaded at all - hence no vyos user.

So it looks like /opt/vyatta/etc/config.boot.default isn’t being copied into the running configuration, not can I find the code that is supposed to do this.

njh · January 28, 2019, 10:23am

Although I haven’t routed any packets yet, I successfully got VyOS 1.2.0 to run on my 32-bit x86 ALIX board last night . I wasn’t sure if I would ever get to this point!

I worked out the reason that there was no vyos user was because the configuration file was failing to load.

This is what was in /var/log/vyatta-config-loader.log:

2000-01-01 01:26:22  == begin boot-config-loader
2000-01-01 01:26:23  -- begin load
DEBUG vexit_internal: get_parsed_tmpl: failed to parse tmpl [/interfaces/wirelessmodem]

/opt/vyatta/sbin/vyatta-boot-config-loader: line 71:  3277 Segmentation fault      $CAPI loadFile $BOOT_FILE
2000-01-01 01:26:23  -- load finished with failure(s)
2000-01-01 01:26:23  -- begin commit
2000-01-01 01:26:23  -- commit succeeded
2000-01-01 01:26:24  -- teardown succeeded
2000-01-01 01:26:24  -- exiting

My solution for now was to delete the wirelessmodem nodes:

rm -Rf /opt/vyatta/share/vyatta-cfg/templates/interfaces/wirelessmodem

Hopefully now that I have a ‘working’ system, I will be able to work out how fix things properly and submit some Pull Requests back.

I have documented what I learnt about the boot process here:
https://wiki.vyos.net/wiki/Boot_steps

hagbard · January 28, 2019, 5:05pm

There is no wirelessmodem set per default in the config, where do you get that from? Is it in /config/config.boot as config item as well?

njh · January 28, 2019, 11:30pm

There is no /config/config.boot at all - this a problem at first boot. It looks like it is a problem with loading the template files, rather than a problem with the configuration - although I don’t really understand what/how VyOS loads those files.

# dpkg -S /opt/vyatta/share/vyatta-cfg/templates/interfaces/wirelessmodem 
vyatta-ipv6-rtradv, vyatta-cfg-qos, vyatta-cfg-firewall, vyatta-cfg-quagga: /opt/vyatta/share/vyatta-cfg/templates/interfaces/wirelessmodem

I don’t have the vyatta-wirelessmodem package installed at all - but it looks like it has been replaced by vyatta-cfg-firewall. Any suggestions on how I work out what is causing the Seg Fault?

On a positive note I have managed to route some packets this evening

hagbard · January 28, 2019, 11:52pm

What shows sudo dpkg -l vyatta-wirelessmodem?
Should show something like: ii vyatta-wirelessmodem 0.1.24+vyos2+current1.
I highly doubt that a package is being replaced, the only replacements I know of are in vyos-replace, but I could be wrong and there are other mechanisms in place too.
So, I suppose the segfault comes from reading the non existent ‘config.boot’, it it repeatable? Like when you reboot, it shows the same error every time? You can restart the services and trace them to see where it hangs before the segfault. From here you should be able to see what happens before the segfault.
I’m going to test amd64 too for the absence of config.boot and see what happens.

njh · January 29, 2019, 12:17am

vyatta-wirelessmodem is not installed:

sudo dpkg -l vyatta-wirelessmodem
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
un  vyatta-wireles <none>       <none>       (no description available)

This commit implies it was replaced by vyatta-cfg-firewall 11 years ago!!

Yes, the segfault is repeatable. New lines are added to the /var/log/vyatta-config-loader.log log file every time I run /usr/libexec/vyos/init/vyos-router start.

Will see if I can get anything meaningful out of strace

hagbard · January 29, 2019, 6:04pm

Something is still wrong, looks like some packages aren’t being installed and I wonder if those are the only ones. I see in the control file the replacements for 3 packages however they are all installed on an amd64 image.
(vyatta-cfg-firewall/control at current · vyos/vyatta-cfg-firewall · GitHub)

on the rolling from yesterday:
ii vyatta-cfg-system 0.20.44+vyos2+current19
ii vyatta-cfg-firewall 0.14.0+vyos2+current2
ii vyatta-openvpn 0.2.60+vyos3+current2
ii vyatta-wirelessmodem 0.1.24+vyos2+current1

The opens a whole bunch of questions :D. Usually they shouldn’t exist on the rolling images as well. I tested the removal of the config.boot file, no issues it’s recreated by a script. I haven’t check which one yet.