VyOS/Vyatta smp_affinity

Development
, , , ,
1

Hi!

Does anyone know why on the same machine (PowerEdge 1850) VyOS (1.1.5) see only two processors - but Vyatta6.5 see four ?

Zlatko

VC6.5:
cat /proc/interrupts | grep eth
37: 0 13704626 0 1 IO-APIC-fasteoi eth2
64: 19240789 0 0 7 IO-APIC-fasteoi eth0
65: 0 0 3031555 10 IO-APIC-fasteoi eth1

VyOS1.1.5:
cat /proc/interrupts | grep eth
16: 1209 0 0 6 IO-APIC-fasteoi uhci_hcd:usb2, eth0
17: 0 2446 1 9 IO-APIC-fasteoi eth1, eth2

sh ver
Version: VC6.5R1
Description: Vyatta Core 6.5 R1
Copyright: 2006-2012 Vyatta, Inc.
Built by: autobuild@vyatta.com
Built on: Fri Nov 16 16:39:16 UTC 2012
Build ID: 1211161646-334fb58
System type: Intel 32bit
Boot via: image
HW model: PowerEdge 1850
sh hardware cpu
Architecture: i686
CPU op-mode(s): 32-bit, 64-bit
CPU(s): 4
Thread(s) per core: 2
Core(s) per socket: 1
CPU socket(s): 2
Vendor ID: GenuineIntel
CPU family: 15
Model: 4
Stepping: 1
CPU MHz: 3391.623
L1d cache: 16K
L2 cache: 1024K

2

VyOS does see 4 CPUs. Are you referring to an affinity for some subset? If so, I have observed that as well - I have no answers.

3

… default - affinity it’s set to auto, I tried to set it on a dedicated value for each interfaces - set interfaces ethernet eth0 smp_affinity 1 … and so on … - but result it’s the same … :frowning:

4

…and the consequence is that Vyatta has a much better performance pps…in may test I sent a single udp/64B stream, 960k pps - Outside to DMZ, and Vyatta forwarded to the DMZ 280k pps while vyos 200k… :frowning:

5

I think you have to test with multiple streams. Only one thread/core/cpu can operate on routing packets for a single stream. I would try more streams to simulate real world routing and see what that does.

6

…of course with more flows - pps score is better (and for vyos and for Vyatta)…but I just compared the Vyatta / vyos on the same machine and the same test … and I noticed vyos not interrupt each NIC to a different cpu ( my question: -why?) … and therefore achieved a worse score pps …