Vyos image updates as code

tomcat667 · June 13, 2024, 5:25am

Hi,

I have bunch of Vyos installations but how to update over a Jenkins Pipline.
First idea was connect to the specific Vyos and execute a command over SSH

#!/bin/vbash
source /opt/vyatta/etc/functions/script-template
echo | /opt/vyatta/bin/vyatta-op-cmd-wrapper add system image /home/jenkins/iso/vyos-rolling-latest.iso

but that works not reliably any idea how do that with ssh correct?
My main issue is when I use the add system image commands, there are 4 questions I mean 3 answers with yes and one was the new name of the image.
There are a options to create options to install the image directly without questions.

greetz

talmakion · June 13, 2024, 5:57am

I’m using a simple lash up via Ansible, vyos.vyos.vyos_config/.vyos_command and ansible.netcommon.cli_command to manage my collection of test VMs:

The playbook can be given a specific URL or find the latest release from the nightly repo via github API
Download it locally on the Ansible worker
Push a copy of the updated ISO to each VM
Run through the add system image process via ansible.netcommon.cli_command (including making sure the old baseline config is copied to the new version)
Trim back image versions (matching a name pattern) beyond a set number, so it doesn’t just fill up for all time
Let each VM reboot and wait for it to come back
Enforce a bunch of standard configuration on the newly installed image: ssh keys, symlinks out of home (I’ve got a small collection of test configs on un-versioned persistent disk, linked to from ~/testing-configs/), installing packages for smoketests and remote editing over SSH

Handling the add image looks a bit like:

# From the task ###
- name: add current system image if not yet available
  ansible.netcommon.cli_command:
    command: "add system image {{ release_info.stdout_lines[0]|basename }}"
    check_all: yes
    prompt:
      - What would you like to name this image
      - Would you like to set the new image as the default one for boot
      - An active configuration was found. Would you like to copy it to the new image
      - Would you like to copy SSH host keys
    answer:
      - "\n"
      - "y"
      - "y"
      - "y"
  notify: reboot into new firmware
  changed_when: yes
  when: not latest_already_loaded
  tags:
    - external_download

# From the handlers ###
- name: reboot vyos
  vyos.vyos.vyos_command:
    commands:
    - command: reboot
      prompt: Are you sure you want to reboot this system
      answer: y
  listen: "reboot into new firmware"

- name: wait for firmware reboot to complete
  wait_for_connection:
    delay: 25
  listen: "reboot into new firmware"

That’s how I’ve been tracking the current rolling image across ~4 test VMs and I can side-load custom image builds manually if I need to. I’ve got a separate playbook to handle a setup from scratch (booted off an ISO, before it’s even installed to disk) but that one isn’t fully automatable with the state of the Ansible VyOS collection.

I do have a Jenkins to handle custom builds as well (I try not to use it - customising a rolling is a lot quicker) but I had intended to extend it to allow pushing a custom build directly to the test VMs via my playbooks.

My playbooks aren’t publicly accessible, but if you think they’d be handy to look at, I can share the relevant pieces (like install image and more detail on add image handling).

My test VMs are already preconfigured with a management VRF and appropriate routing so that Ansible can reach them over SSH. If deploying from scratch, it might be an idea to look further into something like cloud-init.

echowings · June 13, 2024, 6:01am

jenkins call ansible to upgrade Vyos?

tomcat667 · June 17, 2024, 5:15am

Hi,
ansible is the only way? I have to think about it first.
thx for the hint

talmakion · June 17, 2024, 6:47am

It’s not the only way, but it is something that has decent pre-existing tooling for what you’re trying to do.

You could automate the same steps using some hand-rolled expect scripts (with a bit more effort than Ansible) or use a more familiar orchestration tool, of which there are many.

I’ve used Ansible extensively for other things, so it was the logical choice to automate my VyOS testing setup as well.

tomcat667 · June 17, 2024, 9:56am

Okay the other solution is use the API but the API can not be restrict to allow only the update command only restrict the IP range.

syncer · June 17, 2024, 11:17am

That is a good candidate for a feature request

echowings · June 17, 2024, 11:35am

Upgrade vyos and set reboot task schedule. This works well for me.

---
- name: VyOS - install new image and schedule machine reboot.
  gather_facts: false
  hosts: vyos

  vars:
    vyos_version: 1.3.7
    vyos_dir: "files/"
    vyos_file: "vyos-{{vyos_version}}-amd64.iso"
    vyos_url: "https://xxx/1.3.7/generic-iso/xxx/vyos-1.3.7-amd64.iso"
  tasks:
    # - name: Get config for VyOS devices
    #   vyos_facts:
    #     gather_subset: '!config'

    - name: "Check whether VyOS {{ vyos_version }}  Installed or Not"
      cli_command:
        command: 'show system image | grep {{ vyos_version }} | wc -l'
      register: check_installed_value
      ignore_errors: True

    - name: "debug"
      debug: var=check_installed_value.stdout

    # - name: "Copying {{vyos_file}} to system"
    #   net_put:
    #     src: "{{ vyos_dir }}{{ vyos_file }}"
    #     dest: "{{ vyos_file }}"
    #   when: ansible_net_version  != "VyOS 1.3.3"


    - name: "Download vyos {{ vyos_version }} iso"
      cli_command:
        command: curl -fsSLO {{ vyos_url }}
      # when: ansible_net_version  != "VyOS 1.3.3"
      when: check_installed_value.stdout != "1"


    - name: "Installing {{ vyos_version }} on system"
      cli_command:
        # command: add system image "{{ vyos_file }}"
        command: add system image {{ vyos_file }}
        newline: True
        check_all: True
        prompt:
        - 'What would you like to name this image'
        - 'directory and config file'
        - 'current configuration'
        answer:
        - "{{ vyos_version }}"
        - 'Yes'
        - 'Yes'
      # when: ansible_net_version  != "VyOS 1.3.3"
      when: check_installed_value.stdout != "1"



    - name: Schedule reboot around 04:30 tonight
      cli_command:
        command: "reboot at 04:{{ range(25,35) | random }}"
      when: check_installed_value.stdout != "1"


    - name: Cleanup installation file
      cli_command:
        command: "rm vyos*.iso"
      # when: ansible_net_version  != "VyOS 1.3.3"
      when: check_installed_value.stdout != "1"

jagekurt · June 18, 2024, 8:37pm

With your inspiration I created a playbook which automatically pulls the latest 1.5 rolling release version and performs an update to that version. It also performs a backup of your configuration to a remote host.

Please note that this is only for 1.5 rolling releases!

It also requires that your VyOS router is internet facing to be able to directly download the iso from github.

---
- name: VyOS - Install new rolling 1.5 image and schedule machine reboot
  gather_facts: false
  hosts: vyos

  vars:
    backup_ssh_username: ""
    backup_ssh_password: ""
    backup_ssh_host: ""
    backup_destination_path: "" #Add the absolut path and start with a /
    backup_destination_filename: ""

  tasks:  

    - name: "Get Latest Release from Github"
      community.general.github_release:
        user: vyos
        repo: vyos-rolling-nightly-builds
        action: latest_release
      register: version
      delegate_to: localhost

    - name: Print Latest Version
      ansible.builtin.debug:
        var: version.tag
      delegate_to: localhost

    - name: "Check whether VyOS {{ version.tag }} is Installed or Not"
      cli_command:
        command: 'show system image | grep {{ version.tag }} | wc -l'
      register: check_installed_value
      changed_when: check_installed_value.stdout == "0"
      ignore_errors: True

    - name: "Backup VyOS Configuration"
      cli_command:
        command: "{{ item }}"
        newline: True
        check_all: True
        prompt:
        - 'Do you wish to continue'
        answer:
        - 'Yes'
      loop:
        - configure
        - save scp://{{ backup_ssh_username }}:{{ backup_ssh_password }}@{{ backup_ssh_host }}:/{{ backup_destination_path }}/{{ backup_destination_filename }}
        - exit
      when: check_installed_value.stdout != "1"

    - name: "Installing {{ version.tag }} on system"
      cli_command:
        command: add system image https://github.com/vyos/vyos-rolling-nightly-builds/releases/download/{{ version.tag }}/vyos-{{ version.tag }}-amd64.iso
        newline: True
        check_all: True
        prompt:
        - 'What would you like to name this image'
        - 'Would you like to set the new image as the default one for boot'
        - 'An active configuration was found. Would you like to copy it to the new image'
        - 'Would you like to copy SSH host keys'
        answer:
        - "{{ version.tag }}"
        - 'Yes'
        - 'Yes'
        - 'Yes'
      when: check_installed_value.stdout != "1"
      vars:
        ansible_command_timeout: 60

    - name: Schedule reboot around 04:30 tonight
      cli_command:
        command: "reboot at 04:{{ range(25,35) | random }}"
      when: check_installed_value.stdout != "1"

echowings · June 18, 2024, 10:03pm

I configed IAC with ansible to deploy vyos. So I totally no need back step. Also I implement IAC with terraform/pulumi to deploy vyos with cloudinit.

jagekurt · June 19, 2024, 9:11am

Thats cool. I started created a ansible playbook to manage my entire VyOS configuration, and then VyOS team changed the firewall syntax and I decided to take a break from it. Might start it up again. I used jinja to create my configuration files and then load them into VyOS.

I run my VyOS router as baremetal, and have no need to automatic deployment of VyOS just yet. I am only running VyOS at home.