IKEv2 Remote Access VPN breaks after Let's Encrypt cert renewal — "IKE authentication credentials are unacceptable" (Windows error 13801)

VyOS version: 1.5 (circinus) Symptom: Windows native IKEv2 clients suddenly fail to connect with error 13801 (“IKE authentication credentials are unacceptable”), while the VyOS side looks completely healthy — valid cert, valid time sync, correct config, swanctl --list-certs shows no errors.

Posting this because it cost me several days to fully diagnose, and I couldn’t find existing threads connecting all the pieces. Sharing the full path in case it saves someone else the time.


Setup

  • Two independent VyOS 1.5 firewalls acting as IKEv2 remote-access VPN servers (ra-rw connection type), each on a different ISP

  • Server authentication: x509 certificate issued via VyOS’s built-in ACME/Let’s Encrypt integration (set pki certificate <name> acme ...)

  • Client authentication: EAP-MSCHAPv2 (username/password), standard Windows 10/11 native IKEv2 client

  • Both firewalls had been running stable for months on Let’s Encrypt certs

First red herring: power outage / NTP

The initial trigger for investigating was a power outage that knocked out both firewalls overnight. On restart, VPN connections failed intermittently with the same 13801 error. In that specific case, the actual cause was NTP — the servers came up with drifted clocks before NTP had synced, and IKEv2 certificate validation is extremely time-sensitive. Once NTP settled, that particular episode resolved itself. Worth checking first (show ntp, show date), but in our case it masked a second, unrelated, and more persistent problem — see below.

The real problem: Let’s Encrypt’s Gen-Y intermediate rotation

A few days later, VPN connections started failing again with the identical 13801 error, despite NTP being clean (sub-millisecond offsets). This time the certificate itself was the issue.

Let’s Encrypt has introduced a new certificate hierarchy — “Generation Y” — with new roots ISRG Root YR / ISRG Root YE, alongside the long-established Generation X (ISRG Root X1). New leaf certs are now being issued under intermediates YR1/YR2, chaining to Root YR, which is not yet in most operating systems’ trust stores, including Windows.

Let’s Encrypt does cross-sign back to the trusted ISRG Root X1, but that requires serving the cross-signed intermediate, not just the direct Root YR-signed one. Our VyOS-managed ACME certs were only presenting the direct (untrusted-root) chain.

Confirming it

sudo swanctl --list-certs

Compare the leaf’s authkeyId against the loaded CA’s subjkeyId — if you have a valid chain, they must match exactly.

In our case, after a renew certbot force, the leaf correctly renewed to YR2/YR1, but the CA object VyOS had stored (AUTOCHAIN_<name>) sometimes still contained the previous intermediate. show pki certificate <name> reported CA Present: Yes, which is misleading — it only checks that a CA object reference exists, not that it’s the cryptographically correct one for the current leaf.

Two separate bugs stacked on top of this

  1. VyOS config/disk sync bug: after renew certbot force, the CA object in the config tree could be correct while the on-disk file strongSwan actually reads (/etc/swanctl/x509ca/AUTOCHAIN_<name>.pem) remained stale, surviving even a reboot. Compare:

    show configuration commands | grep "AUTOCHAIN_<name> certificate" | awk -F"'" '{print $2}' | base64 -d | openssl x509 -noout -subject
    sudo cat /etc/swanctl/x509ca/AUTOCHAIN_<name>.pem | openssl x509 -noout -subject
    
    

    If these disagree, forcing a trivial real config change (e.g. add then remove a description field on the CA object) and committing triggered VyOS to regenerate the file correctly.

  2. strongSwan only loads the first certificate from a CA file/value — confirmed via strongswan/strongswan#3072. This means even once you have the correct chain, you cannot cram intermediate + cross-signed root into a single pki ca value — they need to be loaded as separate CA objects. Several people on the Let’s Encrypt community forum hit exactly this (thread 1, thread 2) and resolved it by splitting the cross-signed root into its own file/CA slot.

What actually fixed it for us

After fighting the config-sync bug and the multi-cert-per-CA-file limitation for a while, we decided the ACME/ISRG-YR transition churn wasn’t worth continuing to chase, and switched both firewalls to a commercial DV certificate from a standard reseller (SSL.com), replacing the Let’s Encrypt/ACME setup entirely.

This sidesteps the problem completely: an established commercial root has been in every OS trust store for years, there’s no Gen-Y transition to worry about, and — since the reseller only supplied a two-certificate chain (single intermediate + self-signed root, no root needed to be served) — it also avoids the strongSwan multi-cert-per-file bug entirely.

Note: as of March 2026, CA/Browser Forum rules cap public cert validity at ~200 days regardless of CA, so even a “2-year” commercial purchase is delivered as a subscription that reissues every ~197 days rather than one long-lived cert. Still far more predictable than chasing ACME automation gaps.

Quick outline of the manual cert deployment (if useful to anyone)

# Generate key + CSR
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -subj "/C=xx/O=YourOrg/CN=vpn.example.com"

# Submit CSR to your CA/reseller of choice, get back:
#   server.crt         (your signed leaf cert)
#   server.ca-bundle    (intermediate [+ root, self-signed])

# Split the bundle and confirm which entry is the intermediate (subject != issuer)
csplit -z -f ca_ -b "%02d.pem" server.ca-bundle '/-----BEGIN CERTIFICATE-----/' '{*}'
for f in ca_*; do openssl x509 -in "$f" -noout -subject -issuer; done

# Inside a single `configure` session (VyOS: bash vars don't survive into config mode!)
configure
CERT_B64=$(awk '/BEGIN CERTIFICATE/{flag=1;next}/END CERTIFICATE/{flag=0}flag' server.crt | tr -d '\n')
KEY_B64=$(awk '/BEGIN.*PRIVATE KEY/{flag=1;next}/END.*PRIVATE KEY/{flag=0}flag' server.key | tr -d '\n')
CA_B64=$(awk '/BEGIN CERTIFICATE/{flag=1;next}/END CERTIFICATE/{flag=0}flag' ca_00.pem | tr -d '\n')  # the intermediate only

set pki certificate my_commercial certificate "$CERT_B64"
set pki certificate my_commercial private key "$KEY_B64"
set pki ca my_commercial_ca certificate "$CA_B64"

set vpn ipsec remote-access connection rw authentication x509 certificate 'my_commercial'
set vpn ipsec remote-access connection rw authentication x509 ca-certificate 'my_commercial_ca'
commit
save

Then sudo swanctl --load-creds and verify with sudo swanctl --list-certs that the leaf’s authkeyId matches the CA’s subjkeyId.

Summary / TL;DR

  • 13801 on Windows IKEv2 clients almost always means a broken certificate chain (or, less commonly, clock skew)

  • If you’re on VyOS ACME/Let’s Encrypt certs and this started recently: suspect the Gen-Y (YR1/YR2Root YR) transition

  • Verify with swanctl --list-certs: leaf authkeyId must equal loaded CA subjkeyId

  • renew certbot force may fix the leaf but not reliably sync the CA chain file — verify config vs. /etc/swanctl/x509ca/*.pem directly, don’t trust CA Present: Yes alone

  • strongSwan only reads the first cert per CA file/value — multi-cert chains need separate CA objects (also see strongswan/strongswan#3072)

  • If you want to stop fighting this entirely: a commercial DV cert on an established root sidesteps all of the above

Happy to answer questions if anyone else is hitting this — took a while to pin down all the moving pieces.


References

Hi @hook.ua thank you for writing this issue up in such a nicely manner. Fun fact - exctly this happened to me last week when the intermediate switched from R12 to YR1.

For VPN I would personally setup my own CA and include that as a trusted CA on the clients (and the “server”).

This way you can not only select the validtime of your choice (several public CA’s will most likely soon shrink to 90 days or so due to reasons mandated by Google and others) but also you (and not somebody else) are in control of signing the certs being used for your encrypted VPN-tunnels.

Will the Windows Client recognize such CA/Cetificate?

Sure, thats how a majority of enterprises around here does it when it comes to not only Windows but more or less any OS including phones.

Public CA’s are only used for things like public services such as the company homepage and email over Internet where you cant really tell the client(s) to install your custom CA as a trusted CA (well technically you could tell them but I would expect that most would refuse :wink:

Another usecase is for MDM of phones since you get into a chicken or the egg situation (if you want a phone out of the box to be able to be provisioned using your MDM you must use one of the public CA’s the phone comes preloaded with from the factory) but once the device is managed you push your own CA to be used for internal services since more or less all servers will use SSL/TLS and by that needing a servercert to be used.

Again my assumption is that you use this encrypted VPN for your own devices (no matter if you are a person or a company). Of course this option isnt really valid if its a public VPN-service you are going to provide (well it can still be - there will just be an additional step along with installation of VPN-client but instead of asking the client to install the CA you would just ask them to add trust for the server cert being used).

Examples:

https://www.thewindowsclub.com/manage-trusted-root-certificates-windows

I see, thanks.

I was one of intermediate idea to deploy own CA, but paid public service prevail by simplicity.
Having the working example could easy transition, no doubt.
But expectation of playing with Microsoft Entra and Azure fully diverts me.

I’ve paid just for two years of public CA service )

There is also a chance Microsoft finally would accept new Gen-Y chain from Let’s Encrypt, and i would be able to return old scheme with automated regular updates.

If you are a paying customer to Microsoft then file a support case and ask them to update the list of trusted CA’s for their services?

Not uncommon that this is being missed when Letsencrypt and others suddently add or change their CA’s.

One could of course argue that a company of Microsofts size should be able to fix this on their own and keep track of CA’s being added/removed without you as a customer having to file a support case each time.

Yes, we have subscribed to MS 365, but never seen this subscription as the ability to complain MS for his failure :slight_smile:

Nice option, would be interest to test it.

You can hint them to keep track of: