After long time of excellent performance we have observed strange behavior in our external line so 300Mbps upload but only 10 Mbps download.
AMD hardware is HP Proliant DL385 G7 , centos 7, RouterVM has allocated 4GB ram and 4 cores in KVM,
FRT is received from our external provider but not distributed internally as OSPF default is always announced. Average load on the host is small.
Any idea what is the cause of the asymmetry ?
1.1.x EOL, but try check at first interrupts.
cat /proc/interrupts
With physical line all ok?
same behavior on main (kvm1) and backup lines ( kvm2) had to deploy vyos on bear metal
for diff. reason but problem solved ?! . I’ll take up your suggestion but now there is no traffic load. What row/col in the proc/nterrupts should be worrying ?
Row with entry ethX Each interrupts must be binded to own CPU core. And interrupt counters for all cores must be ± same. Can you share screenshot top command and press 1.
brought back the virtual border router
noc@sbr1:~
noc@sbr1:~
noc@sbr1:~$ top
top - 21:36:18 up 3 min, 1 user, load average: 1.09, 0.53, 0.22
Tasks: 116 total, 1 running, 115 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.0%us, 0.0%sy, 0.0%ni, 96.8%id, 0.0%wa, 0.0%hi, 3.2%si, 0.0%st
Cpu1 : 0.0%us, 0.7%sy, 0.0%ni, 83.8%id, 0.0%wa, 0.0%hi, 15.5%si, 0.0%st
Cpu2 : 0.0%us, 0.0%sy, 0.0%ni, 87.2%id, 0.0%wa, 0.0%hi, 12.5%si, 0.3%st
Cpu3 : 0.0%us, 0.0%sy, 0.0%ni, 96.8%id, 0.0%wa, 0.0%hi, 3.2%si, 0.0%st
Cpu4 : 0.0%us, 0.0%sy, 0.0%ni, 89.8%id, 0.0%wa, 0.0%hi, 10.2%si, 0.0%st
Cpu5 : 0.0%us, 0.3%sy, 0.0%ni, 92.0%id, 0.0%wa, 0.0%hi, 7.3%si, 0.3%st
Cpu6 : 0.4%us, 0.0%sy, 0.0%ni, 74.4%id, 0.0%wa, 0.0%hi, 24.9%si, 0.4%st
Cpu7 : 0.0%us, 0.0%sy, 0.0%ni, 95.4%id, 0.0%wa, 0.0%hi, 4.3%si, 0.4%st
Mem: 8010748k total, 1386824k used, 6623924k free, 25200k buffers
Swap: 0k total, 0k used, 0k free, 94544k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31 root 20 0 0 0 0 S 25 0.0 0:03.95 ksoftirqd/6
3 root 20 0 0 0 0 S 20 0.0 0:04.76 ksoftirqd/0
23 root 20 0 0 0 0 S 18 0.0 0:03.48 ksoftirqd/4
11 root 20 0 0 0 0 S 17 0.0 0:05.03 ksoftirqd/1
15 root 20 0 0 0 0 S 14 0.0 0:04.34 ksoftirqd/2
27 root 20 0 0 0 0 S 13 0.0 0:02.99 ksoftirqd/5
35 root 20 0 0 0 0 S 8 0.0 0:03.65 ksoftirqd/7
19 root 20 0 0 0 0 S 7 0.0 0:04.24 ksoftirqd/3
2285 quagga 20 0 470m 432m 1352 S 2 5.5 0:19.33 bgpd
7 root 20 0 0 0 0 S 1 0.0 0:00.60 rcu_sched
516 root 39 19 0 0 0 S 1 0.0 0:00.37 khugepaged
3618 noc 20 0 19164 1296 948 R 1 0.0 0:00.36 top
656 root 20 0 0 0 0 S 0 0.0 0:00.05 kworker/1:1
2275 quagga 20 0 298m 262m 748 S 0 3.4 0:30.52 zebra
2281 quagga 20 0 40900 2508 1192 S 0 0.0 0:00.08 ospfd
1 root 20 0 8448 808 676 S 0 0.0 0:01.00 init
2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd
4 root 20 0 0 0 0 S 0 0.0 0:00.01 kworker/0:0
5 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/0:0H
6 root 20 0 0 0 0 S 0 0.0 0:00.10 kworker/u16:0
8 root 20 0 0 0 0 S 0 0.0 0:00.00 rcu_bh
9 root RT 0 0 0 0 S 0 0.0 0:00.01 migration/0
10 root RT 0 0 0 0 S 0 0.0 0:00.07 migration/1
12 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/1:0
13 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/1:0H
14 root RT 0 0 0 0 S 0 0.0 0:00.01 migration/2
16 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/2:0
17 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/2:0H
18 root RT 0 0 0 0 S 0 0.0 0:00.01 migration/3
20 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/3:0
21 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/3:0H
22 root RT 0 0 0 0 S 0 0.0 0:00.51 migration/4
24 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/4:0
25 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/4:0H
26 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/5
28 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/5:0
29 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/5:0H
30 root RT 0 0 0 0 S 0 0.0 0:00.05 migration/6
32 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/6:0
33 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/6:0H
34 root RT 0 0 0 0 S 0 0.0 0:00.35 migration/7
36 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/7:0
noc@sbr1:~ ^C
noc@sbr1:~
noc@sbr1:~
noc@sbr1:~
noc@sbr1:~
noc@sbr1:~ top
@Dmitry virtual router now has ram from 4 to 8 , vcpu from 4 cores to 8 to match the physical vyos that
does not show the said asymmetry above
So, do you still have issue on virtual router?
I think problem not with interrupts or cpu/ram, in top output all ok.
I will try to catch again the problem “live” and re-post findings, thank you for your time. The initial question was to get ideas of possible causes of the asymmetry rather than debug the live problem. At the time I found a temp solution to run vyos on bare metal.