I am using VyOS 1.3.x and 1.2.x (the later one less and less) and use VyOS to connect network sites with IPsec tunnels (with and without inner GRE-Tunnels). Up to now VyOS only support RSA keys out of the box. With a little patch of the file /opt/vyatta/sbin/vpn-config.pl you can enable ECDSA support for IPsec connections - but then it only supports ECDSA not RSA anymore. So a mixed use is not possible. I am not a perl programer to dive into that file and find out if it is possible to rewrite it so that both key types could be supported. It would be great if those modern key types will be supported in a more usable way.
Try to get it working on 1.4/1.5 as it uses python3