VPP is not using isolated cpus (bug?)

General questions
1

Hello everyone,

These days I’m trying to set up vpp on VyOS.

According to documents, I noticed we should use set system option kernel cpu isolate-cpus x,xx-xx to isolate specified CPUs from the kernel scheduler.

But vyos vpp wouldn’t use these isolated cpu at this time (rolling 2026.04.02). The doc (VPP Dataplane CPU Configuration — VyOS rolling release (current)) says:

The system automatically assigns cores using the following rules:

  • The first two CPU cores are always reserved for the operating system and other services.
  • The main VPP thread is assigned to the first available core after the reserved ones.
  • The remaining allocated cores are used for worker threads.

So vpp will always use cpu2 for main thread and cpu3,4,5,… for worker threads.

I think this behavior is not what we expect and should be considered a bug. Specifically, when a user explicitly defines isolate-cpus for the dataplane, the expectation is that VPP will utilize those dedicated resources. However, the current “automatic assignment” logic completely ignores these isolated cores and instead grabs other cores that might be intended for system services. This leads to a critical waste of CPU resources and results in performance degradation.

The correct behavior should not rely on a hardcoded sequence. Instead, we should be able to manually define the CPU core binding for each VPP interface. This is the only way to ensure strict alignment between the CPU and the NIC on the same NUMA node, which is essential to obtain normal performance.

The points mentioned above are based on my recent learning. If there are any inaccuracies in my understanding, please feel free to point them out.

2

Hi! Yes, indeed, there is a little confusion with core assignment in the documentation.

With the current config model, we should select from cores that are isolated, not from all, so I agree - this should be treated as a bug. But the impact is minimal, and the workaround is very simple - just isolate cores starting from 2.

On NUMA systems, everything is way more complicated, and for the sake of truth: we never recommend multi-CPU setups to customers, and there are way more nuances than this one that will impact behavior and performance here and there. So, a short recommendation: simply do not use NUMA if you are not totally sure this is what you need and that it works for your specific case.

But yes, small or big, the bug is fairly real. Could you submit a bug report at https://vyos.dev/?

3

We are using single EPYC CPU for our router. But because the cpu has 4 CCDs, someone recommended me choose NPS4 mode and bind the corresponding core to the NIC. I’m not sure how much performance degradation it will result in if we always choose core2-N as isolated cpus. So I think we may give an option to every nic(at least as an alternative option, the default is choose 2-n)?

I believe those who choose VPP are all chasing peak performance. Given that VPP configuration is inherently complex, users likely won’t mind manually pinning cores to each network interface.

4

Thanks for the details!

I’ve created a bug report to track this: ⚓ T8460 VPP: Inconsistency between VPP cores allocation and CPU isolation configuration

Also, you should use NPS1 to avoid issues.

5

What does NPS1 mean in this context?

6