Vyos KVM QEMU 100% CPU stuck

Creating a quick forum post in the hope this will aid others, it took quite some time to get to the bottom of what now seems like a fairly obvious fix but i couldn’t find much information when googling.

tl;dr check your “host CPU configuration” setting and change it to kvm64"

As the title says I was getting issue where VYOS would become up responsive to any network connectivity (both passing traffic and responding to ping/SSH to the box itself). The console would periodically report " “soft lockup cpu stuck for XXs” and a message about “jiffies”.

Top on the host was reporting QEMU was taking 100% of all allocated CPUs but top on the guest VYOS reported the system was barely idling.

I tried pinning the vCPUs to specific cores, increasing the number of cores, renicing the QEMU process, changing the interface type from the default RTL8139 to virtio and shutting down other processes on the host to reduce overall CPU. Nothing worked and I was about to give up and buy a hardware firewall.

Turns out that KVM wizard had defaulted to copying the host CPU configuration (SandyBridge).
Setting this to “kvm64”, stopping and then starting the virtual machine (Force reset didn’t do it) fixed the issue.

System has now been stable for the last couple of days without any stuck messages and CPU on the host is around 12.5% with around 80mbps of VPN traffic.

Additional Details:
HP ML350 Gen8
Intel® Xeon® CPU E5-2430 0 @ 2.20GHz (VT-x enabled)
32Gb RAM
Ubuntu 18.04
QEMU Version: 1:2.11+dfsg-1ubuntu7.23

2 Likes

Thanks for sharing!
This is really useful info for all who runs VyOS on KVM or RHEV