PKI-based IKEv2/IPSec VPN authentication fails because the intermediate CA's private key isn't present

I’m doing some proof-of-concept work with IKEv2/IPSec and PKI. I’ve imported the public keys for the root and intermediate CAs, and the private and public keys for the router’s certificate. Everything looks good, but authentication fails because VyOS can’t find the private key for the intermediate CA. I’m going to continue working on this myself, but this seems like a really odd problem as VyOS shouldn’t have/need the private keys for the CAs at all. I’m using 1.4-rolling-202301280924 in the lab and here’s what I’m getting.

Feb 23 12:32:44 ImportantPerson-Centre charon: 07[NET] <3984> received packet: from 203.0.113.241[4500] to 203.0.113.242[4500] (432 bytes)
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[ENC] <3984> parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) ]
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[IKE] <3984> 203.0.113.241 is initiating an IKE_SA
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[CFG] <3984> selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[IKE] <3984> sending cert request for "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=VPN Intermediate CA"
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[IKE] <3984> sending cert request for "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=VPN CA"
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[IKE] <3984> sending cert request for "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=VPN Intermediate CA"
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[ENC] <3984> generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(FRAG_SUP) N(CHDLESS_SUP) N(MULT_AUTH) ]
Feb 23 12:32:44 ImportantPerson-Centre charon: 07[NET] <3984> sending packet: from 203.0.113.242[4500] to 203.0.113.241[4500] (521 bytes)
Feb 23 12:32:44 ImportantPerson-Centre charon: 08[NET] <3984> received packet: from 203.0.113.241[4500] to 203.0.113.242[4500] (1236 bytes)
Feb 23 12:32:44 ImportantPerson-Centre charon: 08[ENC] <3984> parsed IKE_AUTH request 1 [ EF(1/2) ]
Feb 23 12:32:44 ImportantPerson-Centre charon: 08[ENC] <3984> received fragment #1 of 2, waiting for complete IKE message
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[NET] <3984> received packet: from 203.0.113.241[4500] to 203.0.113.242[4500] (756 bytes)
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[ENC] <3984> parsed IKE_AUTH request 1 [ EF(2/2) ]
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[ENC] <3984> received fragment #2 of 2, reassembled fragmented IKE message (1520 bytes)
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[ENC] <3984> parsed IKE_AUTH request 1 [ IDi AUTH CERT N(INIT_CONTACT) SA TSi TSr ]
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[IKE] <3984> received end entity cert "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=Town Hall"
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <3984> looking for peer configs matching 203.0.113.242[%any]...203.0.113.241[C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=Town Hall]
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984> selected peer config 'town-hall'
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984>   using certificate "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=Town Hall"
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984>   using trusted intermediate ca certificate "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=VPN Intermediate CA"
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984>   using trusted ca certificate "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=VPN CA"
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984>   reached self-signed root ca with a path length of 1
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984> checking certificate status of "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=Town Hall"
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984> certificate status is not available
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984> checking certificate status of "C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=VPN Intermediate CA"
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[CFG] <town-hall|3984> certificate status is not available
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[IKE] <town-hall|3984> authentication of 'C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=Town Hall' with RSA signature successful
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[IKE] <town-hall|3984> no private key found for 'C=CA, ST=Ontario, L=Labville, O=Town of Labville, CN=VPN Intermediate CA'
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[ENC] <town-hall|3984> generating IKE_AUTH response 1 [ N(AUTH_FAILED) ]
Feb 23 12:32:44 ImportantPerson-Centre charon: 09[NET] <town-hall|3984> sending packet: from 203.0.113.242[4500] to 203.0.113.241[4500] (80 bytes)
pki {
    ca vpnCa {
        certificate ****************
    }
    ca vpnIntermediateCa {
        certificate ****************
    }
    certificate vpnCertificateImportantPersonCentre {
        ****************
        private {
            key ****************
        }
    }
}
vpn {
    ipsec {
        esp-group vpnIpsecEspToL {
            lifetime 1800
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes256
                hash sha256
            }
        }
        ike-group vpnIpsecIkeToL {
            dead-peer-detection {
                action restart
                interval 30
                timeout 120
            }
            key-exchange ikev2
            lifetime 86400
            proposal 1 {
                dh-group 14
                encryption aes256
                hash sha256
            }
        }
        interface eth0
        site-to-site {
            peer town-hall {
                authentication {
                    mode x509
                    x509 {
                        ca-certificate vpnIntermediateCa
                        certificate vpnCertificateImportandPersonCentre
                    }
                }
                connection-type initiate
                description "Important Person Centre to Town Hall VPN"
                ike-group vpnIpsecIkeToL
                ikev2-reauth inherit
                local-address 203.0.113.242
                remote-address 203.0.113.241
                tunnel 192 {
                    esp-group vpnIpsecEspToL
                    local {
                        prefix 172.31.195.255/32
                    }
                    protocol gre
                    remote {
                        prefix 172.31.192.255/32
                    }
                }
            }
        }
    }
}

Update: Upgraded the node to 1.4-rolling-202302150317 and the same symptoms are showing.

Went back to 1.3.2, pulled the Intermediate CA out of the configuration and just put in the public key for the Root CA. That seems to have brought up the tunnel. I really like the PKI configuration in 1.4, but there’s still some strangeness there.

Does it work if you update from 1.3 to 1.4?
All migration scripts should work during update

Please note the format of the private key changed in 1.4 from PKS#1 to PKS#8. Info = cryptography - PKCS#1 and PKCS#8 format for RSA private key - Stack Overflow