I use a Protectli Vault for my home router/firewall with VyOS. I have 1Gbps fiber internet which has a fiber box in my home that I plug directly in to my router.
When I plug my laptop straight into the fiber box, I get dang near 1Gbps each way on speedtests. Greater than 950 each direction.
I have my router plugged into a 24 port unifi gigabit switch, and then many things on my switch. When I run a speedtest from terminal on a server plugged directly in to the switch, I get these speeds on average:
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted [26.37 km]: 12.628 ms
Testing download speed................................................................................
Download: 877.38 Mbit/s
Testing upload speed......................................................................................................
Upload: 732.62 Mbit/s
My question… I installed speedtest client in when I built VyOS. From the VyOS terminal, I get this, every single time:
Why the seriously slow download speeds? I expect some loss from direct to fiber, because I have many things on my network doing things. So the loss in speed from fiber box to the server I listed first is expected. But this download speed of mid to low 200’s is odd to me.
I watch the cpu during the test, nothing max’s out. Yeah, I see it hit when running, but each core is under 100%.
Thanks for the tip. I tried to read up on that, not sure I really get it.
Either way, I enabled gso/gro/tso one at a time and tested between each. No change at all in speed.
I just don’t know why up is so fast compared to down.
Possible, but isn’t it likely that then all clients on the network would have the issue? Machines that go through this router don’t have the problem. Just the router itself.
Could be just about anything from the transceiver to the cable itself at the VyOS end or in the remote end which the cable is connected to.
I would try to liveboot latest ubuntu (23.04 as of writing) on the box and run the speedtest to compare to see if the result is similar or better/worser.
When it comes to settings of the VyOS itself this is what you can verify (one block at a time and reboot in between):
set firewall global-options all-ping 'enable'
set firewall global-options broadcast-ping 'disable'
set firewall global-options ip-src-route 'disable'
set firewall global-options ipv6-receive-redirects 'disable'
set firewall global-options ipv6-source-validation 'strict'
set firewall global-options ipv6-src-route 'disable'
set firewall global-options log-martians 'enable'
set firewall global-options receive-redirects 'disable'
set firewall global-options resolver-cache
set firewall global-options resolver-interval '60'
set firewall global-options send-redirects 'disable'
set firewall global-options source-validation 'strict'
set firewall global-options syn-cookies 'enable'
set firewall global-options twa-hazards-protection 'disable'
set interfaces ethernet ethX ip arp-cache-timeout '240'
set interfaces ethernet ethX offload gro
set interfaces ethernet ethX offload gso
set interfaces ethernet ethX offload lro
set interfaces ethernet ethX offload rfs
set interfaces ethernet ethX offload rps
set interfaces ethernet ethX offload sg
set interfaces ethernet ethX offload tso
set interfaces ethernet ethX ring-buffer rx '4096'
set interfaces ethernet ethX ring-buffer tx '4096'
set system conntrack expect-table-size '10485760'
set system conntrack hash-size '10485760'
set system conntrack log icmp new
set system conntrack log other new
set system conntrack log tcp new
set system conntrack log udp new
set system conntrack table-size '10485760'
set system conntrack timeout icmp '10'
set system conntrack timeout other '600'
set system conntrack timeout tcp close '10'
set system conntrack timeout tcp close-wait '30'
set system conntrack timeout tcp established '600'
set system conntrack timeout tcp fin-wait '30'
set system conntrack timeout tcp last-ack '30'
set system conntrack timeout tcp syn-recv '30'
set system conntrack timeout tcp syn-sent '30'
set system conntrack timeout tcp time-wait '30'
set system conntrack timeout udp other '600'
set system conntrack timeout udp stream '600'
set system ip arp table-size '32768'
set system ip disable-directed-broadcast
set system ip multipath layer4-hashing
set system ipv6 multipath layer4-hashing
set system ipv6 neighbor table-size '32768'
set system option performance 'throughput'
set system sysctl parameter kernel.core_uses_pid value '1'
set system sysctl parameter kernel.sysrq value '0'
set system sysctl parameter vm.swappiness value '1'
set system sysctl parameter vm.vfs_cache_pressure value '50'
Note not all above are necessary performance related, some are just sane defaults from my point of view.
And there is work in progress to get VPP (aka Intel DPDK) available which would dramatically improve performance but that is currently at “alpha stage” or what one would call it.
Historically the main reason for performance degradation between older 1.3 and newer 1.3, 1.4 and 1.5 have been that some of the offload settings (per interface) which previously were enabled by default in the linux kernel no longer is. That is you must manually enable that.
Also note that not all offloading options are valid for your card(s) and some can also be harmful but generally speaking enable them all “should” be safe.
Also note that you should reboot when you enable/disable the settings (normally shouldnt be needed but “did you turn it off and on again?” applies
