TRO offloading kills throughput

General questions
1

Hello All

New here. I’m looking for a virtualized firewall solution (XCP/XEN on HP edge line servers) so I have been looking into Vyos with some success so far. I’m on the latest build and I need some recommendations for interfaces settings on a virtualized setup. I noticed TSO offloading was on by default killing my throughput (3-7Mbps) so I disabled TSO and and SG offloading. I’m now getting 200-250Mbps in a 1gig/30 internet connection. Is there anything I can do to improve this? I’m getting full bandwidth on the outside interface?

Here is my current config:

ethernet eth0 {
address 10.254.250.1/24
description INSIDE
offload {
gro
gso
}
}
ethernet eth1 {
address dhcp
description OUTSIDE
offload {
gro
gso
}
}

Speed test on device:

Idle Latency: 24.77 ms (jitter: 18.59ms, low: 16.17ms, high: 53.99ms)
Download: 938.59 Mbps (data used: 1.0 GB)
23.48 ms (jitter: 6.41ms, low: 13.33ms, high: 263.13ms)
Upload: 54.92 Mbps (data used: 93.7 MB)
14.40 ms (jitter: 3.01ms, low: 6.31ms, high: 111.99ms)
Packet Loss: 0.0%

2

Yes, depending on the vNIC you choose, you can check the ring-buffer:

set interfaces ethernet ethX ring-buffer
Possible completions:
   rx                   RX ring buffer
   tx                   TX ring buffer

check the multi-quee on vNIC

sudo ethtool -l ethX

and flowtables in your configuration , it should help to increase your performance :

https://docs.vyos.io/en/latest/configuration/firewall/flowtables.html#overview

3

Here you got some:

4

Thank you. I’m figuring out that much of the interface offloading must be handled on the Dom0 level. For reasons that are well beyond my understanding this is the best I can come up with so far.

Xen Dom0 Level settings ($uuid is the physical interface uuid):

xe pif-param-set uuid=$uuid other-config:ethtool-gso=“on”
xe pif-param-set uuid=$uuid other-config:ethtool-ufo=“off”
xe pif-param-set uuid=$uuid other-config:ethtool-tso=“off”
xe pif-param-set uuid=$uuid other-config:ethtool-sg=“off”
xe pif-param-set uuid=$uuid other-config:ethtool-tx=“off”
xe pif-param-set uuid=$uuid other-config:ethtool-rx=“off”

Vyos interface settings:

ethernet ethx {
address dhcp
description OUTSIDE
offload {
gro
gso
}