How to properly build and smoketest VyOS?

Development
1

I tried following the instructions at Build VyOS — VyOS 1.4.x (sagitta) documentation but I must be missing something here?

I noticed that the iso that got built didnt include my “T1234” branch with fixes at all but the original older files were still in the iso, so what did I do wrong?

Anyone in the forum who could write some ELI5 instructions on this topic?

What I want to do:

  1. Before you start make sure to install package docker-ce and then either run the docker (build) using sudo or add current user to docker-group on the system (which it seems as similar risk).

  2. Build my own dockerfile based on GitHub - vyos/vyos-build: VyOS image build scripts (latest current).

Solution?

1.1)

mkdir vyos && cd vyos && git clone -b current --single-branch https://github.com/vyos/vyos-build

1.2)

And then create a /home/username/vyos/vybld.sh with following content:

#!/bin/sh
docker run --rm -it \
    -v "$(pwd)":/vyos \
    -v "$HOME/.gitconfig":/etc/gitconfig \
    -v "$HOME/.bash_aliases":/home/vyos_bld/.bash_aliases \
    -v "$HOME/.bashrc":/home/vyos_bld/.bashrc \
    -w /vyos --privileged --sysctl net.ipv6.conf.lo.disable_ipv6=0 \
    -e GOSU_UID=$(id -u) -e GOSU_GID=$(id -g) \
    vyos/vyos-build:current bash

1.3)
Finally run ./vybld.sh.

Variant of above:

  1. Build my own dockerfile based on a branch I have created at github for example T1234 at https://github.com/username/vyos-build

Solution?

2.1)

mkdir vyos && cd vyos && git clone -b T1234 --single-branch https://github.com/username/vyos-build

Then the remaining steps are as chapter 1 above?

  1. Build my own ISO to be able to run smoketests, also based on a branch I have created at github for example T1234 at https://github.com/username/vyos-1x

Solution?

3.0)

First do chapter 1 or 2 above.

Then:

3.1)

mkdir vyos && cd vyos && git clone -b 1234 --single-branch https://github.com/username/vyos-1x

3.2)

Smoketest image:

docker run --rm --privileged -v ./vyos-build/:/vyos -w /vyos vyos/vyos-build:current sudo --preserve-env make clean
docker run --rm --privileged -v ./vyos-build/:/vyos -w /vyos vyos/vyos-build:current sudo --preserve-env ./build-vyos-image --architecture amd64 --build-by "[email protected]" --vyos-mirror https://rolling-packages.vyos.net/current/ --debian-mirror http://deb.debian.org/debian/ --build-type release --custom-package vyos-1x-smoketest --version "1337-2023" iso

Release image:

docker run --rm --privileged -v ./vyos-build/:/vyos -w /vyos vyos/vyos-build:current sudo --preserve-env make clean
docker run --rm --privileged -v ./vyos-build/:/vyos -w /vyos vyos/vyos-build:current sudo --preserve-env ./build-vyos-image --architecture amd64 --build-by "[email protected]" --vyos-mirror https://rolling-packages.vyos.net/current/ --debian-mirror http://deb.debian.org/debian/ --build-type release --version "1337-2023" iso

Finally to run the smoketests, boot the iso (do there perhaps already exist a script for this?) and run:

/usr/bin/vyos-smoketest > /home/vyos/smoketest.log

Also do you have to install VyOS on a drive for the smoketests to run properly or is it fine to run it as a live boot image?

Wild guess, is it “vyos/vyos-build:current” which I should change into “vyos/vyos-build:T1234”?

1 Like
2

Thanks to Simon at the slack-channel!

Here is some documentation in case somebody else stumbles upon this in future:

  1. Prerequisites.

0.1) Install docker-ce package:

sudo apt-get update
sudo apt-get install docker-ce

0.2) Put user in docker-group (and preferly logout/login to make this active):

sudo usermod -aG docker username

0.3) Create workdir:

mkdir vyos

0.4) Create /home/username/vyos/vybld.sh (and set permission 755 chmod +x /home/username/vyos/vybld.sh):

#/!bin/sh

docker run --rm -it \
    -v "$(pwd)":/vyos \
    -v "$HOME/.gitconfig":/etc/gitconfig \
    -v "$HOME/.bash_aliases":/home/vyos_bld/.bash_aliases \
    -v "$HOME/.bashrc":/home/vyos_bld/.bashrc \
    -w /vyos --privileged --sysctl net.ipv6.conf.lo.disable_ipv6=0 \
    -e GOSU_UID=$(id -u) -e GOSU_GID=$(id -g) \
    vyos/vyos-build:current bash

To build stuff (dont forget to logout/login to make use of updated groups if you not already did so).

  1. Enter workdirectory:
cd vyos
  1. Clone stuff from github:

'``
git clone -b current --single-branch GitHub - vyos/vyos-build: VyOS image build scripts
git clone -b current --single-branch GitHub - vyos/vyos-1x: VyOS command definitions, scripts, and utilities


Then if you are to test some new code you provide you can edit the content of the vyos-build and vyos-1x directories.

Or if you already sent a commit you can fetch that branch from your github:

'``
git clone -b current --single-branch https://github.com/vyos/vyos-build
git clone -b T1234 --single-branch https://github.com/username/vyos-1x
  1. Start docker:
./vybld.sh

If user not part of docker-group then sudo must be used:

sudo ./vybld.sh

You can see that docker have successfully started by looking at the prompt:

username@fddba08ee50d:/vyos
  1. Build the vyox-1x debian packages (.deb) by entering the vyos-1x folder and execute (while in docker):
cd vyos-1x
dpkg-buildpackage -uc -us -tc -b
  1. Move the .deb files created to vyos-build/packages directory:
mv ../*.deb ../vyos-build/packages/
  1. Enter vyos-build directory and create the ISO image.

6.1) To build release with smoketests:

cd ../vyos-build
./build-vyos-image --architecture amd64 --build-by "[email protected]" --vyos-mirror https://rolling-packages.vyos.net/current/ --debian-mirror http://deb.debian.org/debian/ --build-type release --custom-package vyos-1x-smoketest --version "1337-2023" iso

6.2) To build release without smoketests (remove some of the custom deb-files in vyos-build/packages directory aswell):

cd ../vyos-build
./build-vyos-image --architecture amd64 --build-by "[email protected]" --vyos-mirror https://rolling-packages.vyos.net/current/ --debian-mirror http://deb.debian.org/debian/ --build-type release --version "1337-2023" iso
  1. To run tests (this will start the iso within qemu still being in docker).

7.1) Smoketests:

sudo make test

7.2) Config tests:

sudo make testc

Done!

To use the iso-file elsewhere you can copy it from (example) /home/username/vyos/vyos-build/build/vyos-999.202309182137-amd64.iso (unless you set a custom version to it when calling ./build-vyos-image).

Any improvements?

How safe is it to run the “sudo make test” outside of the docker, Im thinking in order to gain some performance as in host(qemu(vyos(smoketests))) rather than host(docker(qemu(vyos(smoketests))))?

4 Likes
3

Does anyone know if I should be able to run the smoketests, under the qemu emulator, in the pre-built docker container? I’m able to build an ISO image and even run the image on physical hardware, but after I build the iso, I run sudo make test, I get an error about the iso

 INFO - Removing disk file: testinstall-20240906-165148-35cc.img
ERROR - Hmm... system got an exception while processing.
ERROR - The ISO image is not considered usable!
full command output

make_test_error.txt (5.3 KB)

Also, the iso was built with the following command:

 sudo ./build-vyos-image \
    --architecture amd64 \
    --build-by "[email protected]" \
    --build-type=release \
    --custom-package vyos-1x-smoketest \
    --vyos-mirror https://rolling-packages.vyos.net/current/ \
    --debian-mirror http://deb.debian.org/debian/ \
    generic