PKI config gone after reboot VyOS 2025.09.27-0018-rolling

zarianu · September 28, 2025, 2:56am

Hello dear sirs,

I’m seeing strange VyOS behavior in VyOS 2025.09.27-0018-rolling and in 2025.09.17-0018-rolling earlier. After reboot I got warning when enter configuration mode:

WARNING: There was a config error on boot: saving the configuration now could overwrite data.
You may want to check and reload the boot config

When I do “compare commands 0” I see that all PKI section is gone - CAs and ACME certificate. After “load” command I get my config back and after commit I see Let’s Encrypt certificate being getting from scratch. After that everything works like a charm till next reboot.

What diagnostics / logs should I check?

TIA, regards,
Konstantin

c-po · September 28, 2025, 8:27am

@zarianu please share with us your configuration:

show configuration commandsexecuted in op-mode. With this we can replicate you setup and see what’s wrong.

zarianu · September 28, 2025, 12:12pm

config-28092025.txt (11.8 KB)
@c-po please find config attached

hook.ua · September 29, 2025, 4:16pm

Hi,

I have found this issue few months ago but didn’t recive any feedback how to fix or avoid.

I reverted my instalation to early more or less stable verions (in meands of CA/PKI) from april/may 2025.

c-po · September 30, 2025, 6:35pm

Bug confirmed and tracked via ⚓ T7885 pki: configuration issues on reboot when ACME is used together with OpenConnect

c-po · October 2, 2025, 9:49pm

github.com/vyos/vyos-1x

pki: T7885: check_port_availability() can't be used during system boot

current ← c-po:pki-acme-listen-address

opened 09:45PM - 02 Oct 25 UTC

c-po

+28 -3

## Change summary Multiple related fixes for system startup and ACME based …certificates. A call to `check_port_availability()` will always fail during system boot when listen_address is set and the address is not yet assigned to an interface. This happens b/c PKI subsystem is called prior to any interface - e.g. ethernet - and thus the OS will always be unable to `bind()` a `socket()` to a non existing IP address. Assume someone deleted the certbot_config folder, `renew certbot force` alone will not work as there are no configuration files left to know what to renew. Re-run CLI PKI helper to initially request certificates via ACME again. This "should" (famous last words) never be the case - but sometimes the universe has a bad time. If `/config/auth/letsencrypt` is missing, ACME certificate data can't be merged into the VyOS configuration, preventing interface (e.g., ethernet) setup during boot. This change ensures basic IP connectivity is established, allowing remote management instead of leaving the system entirely inaccessible. This is not what we will expect but if a user deletes `/config/auth/letsencrypt` - this is what will happen. ## Types of changes - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Code style update (formatting, renaming) - [ ] Refactoring (no functional changes) - [ ] Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component - [ ] Other (please describe): ## Related Task(s) * https://vyos.dev/T7885 ## Related PR(s) ## How to test / Smoketest result ``` vyos@vyos:~$ sudo rm -rf /config/auth/letsencrypt vyos@vyos:~$ renew certbot force WARNING: Directory "/config/auth/letsencrypt" missing. Reinitializing PKI subsystem... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /config/auth/letsencrypt/renewal/VYOS.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Renewing an existing certificate for VYOS - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Congratulations, all renewals succeeded: /config/auth/letsencrypt/live/VYOS/fullchain.pem (success) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Hook 'post-hook' ran with output: Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done. ``` Also use this CLI config ``` set pki certificate vpn.xxx.net acme domain-name 'r1.vyos.net' set pki certificate vpn.xxx.net acme email 'acme-test@vyos.net' set pki certificate vpn.xxx.net acme listen-address '1.2.3.4' set pki certificate vpn.xxx.net acme url 'https://acme-staging-v02.api.letsencrypt.org/directory' ``` Save the configuration and reboot the system - no config error must happen ## Checklist: - [x] I have read the [**CONTRIBUTING**](https://github.com/vyos/vyos-1x/blob/current/CONTRIBUTING.md) document - [x] I have linked this PR to one or more Phabricator Task(s) - [ ] I have run the components [**SMOKETESTS**](https://github.com/vyos/vyos-1x/tree/current/smoketest/scripts/cli) if applicable - [x] My commit headlines contain a valid Task id - [ ] My change requires a change to the documentation - [ ] I have updated the documentation accordingly