Hi.
I am using vyos in KVM for a home lab and was chasing down the source of a bandwidth problem.
I have a linux guest setup as my iperf client (192.168.10.51/24)
It is directly connected to the vyos eth0 interface (via a bridge of course) . 192.168.10.73/24. I’ll call that my vyos downlink.
Vyos has another interface (uplink): 11.0…3.1/30
Setting vyos in iperf server mode, i test the throughput between the client & vyos and i get, as expected, very good throughput: 7.6 Gbps (it is qemu with vhost protocol)
Now if i set the vyos uplink as my iperf server, so as to test throughput after routing, (Bind iperf server to uplink address), i get a significant drop of throughput. 1.1 Gbps.
Now if i set the iperf test to the uplink in reverse mode, i.e. the vyos uplink is transmitting while my client is on the receiving end, the performances are again 7Gbps.
I am puzzled why routing through Vyos (test 1) caused a drop of throughput in my second test but then the same test in reverse direction (still routing) would show 7Gbps.
At first i was thinking that it could have to do with RSS/TSO,… but in the test where i was testing the downlink, the throughput is good so RSS/LRO, is good and would be the same for the 2nd test when packet are sent to eth0 only to be routed to eth1 (and throughput is 1Gbps).
Now if throughput is slowed down by the routing engine, how come when i initiate from the uplink, performances are great even though we still route?
Am i missing something? misunderstanding how the network datapath is handled by KVM?
Note that i also just did another test. Vyos as client. Linux vm as server.
initiate iperf from downlink to linux appliance: i get about 6.5 Gbps throughput.
initiate iperf from vyos uplink to linux appliance: i get about the same results (6.5 Gbps throughput).
i’m puzzled… Any idea out there?
Thank you!
PS: one thing i forgot to mention. In the iperf test where traffic slowed down to ~1Gbps, we see lots of TCP retransmits being reported by iperf.