Hi all - was wondering if anyone has experience of using Vyos on a PC Engines APU2 board.
Setup is working fine, just was upgraded to a 1gb/s line via FTTH. Connection is made using the ISPs hardware and then using IP passthrough handed off to the Vyos devices via ethernet.
Speed tests showing a max of 600mb/s down and around 300mb/s down.
Does anyone know how I can check if this is a device limitation?
I don’t have any experience with an APU2, but I’ll ask this:
Does your ISP use PPPoE? People never, ever mention this in their posts, yet it’s a huge factor in performance. Having to encapsualate and de-encapsulate every since packet egressing/ingressing into your router will have a big performance impact.
You could check if this is a device limitation by looking at top/htop while running your tests, is the CPU hitting 100%?
There’s some performance tuning notes here.
There’s also notes here about turning off the meltdown/spectre workarounds for extra performance, but reading this post seems to indicate that maybe 600mb/s is the limit of the device (assuming you ARE using PPPoE)
Hope that helps a little?
PS: Thanks for supporting the project as a Patron. (I am not a project member, just a happy user)
Appreciate the quick response and informative links. My search skills clearly need some improvement.
PPPoE, I believe Swisscom do, but that is all done on their device acting as the IP bridge. My line is capable of 10gb/s and I have no doubt the hardware they supply can handle a 1gb/s link with no trouble.
I quickly configured an Edgerouter 6P and enable hardware offloading and get as close to the 1gb/s line speed as possible. 910mb/s. I suspect if I want to continue running Vyos I need some new hardware…
On that note, any recommendations for devices capable of running line speed on a 1gb/s connection that are suitable for the home. I don’t have the space of rack based kit unfortunately.
Having just double-checked, my connection is using IPoE. I don’t think it makes any difference as the APU isn’t doing any encapsulation/decapsulation. It sees a standard IP interface on its WAN port.
Hello @phillipmcmahon, how you run this bandwidth test?
Could you please take a screenshot sudo top command and press 1 when you run this bandwidth test?
Hello @phillipmcmahon, it looks like queues didn’t apply properly for interfaces higher then eth1
Please, edit sudo nano /etc/modprobe.d/igb-options.conf
And change
options igb RSS=0,0,0,0
Then execute
sudo /usr/sbin/update-initramfs -u
After reboot HW interrupts for eth2 should be applied
Applied the queue changes and also the ring buffer.
I assume each time I update the rolling release I have to apply the queue changes?
Gets me up to 750 down / 650 up. I suspect I am hitting the limits of this SBC.
However, that is good enough for me while I investigate another higher capacity SFF to run Vyos, rather than going back to EdgeOS.
Another major influencing factor is the support on this platform. Active and helpful by users AND developers. Something that can no longer be said about the Edgemax forums.
Any recommendations for home use hardware (not noisy rack mount stuff) that can drive a 1gb/s line at full speed in both directions would be greatly appreciated.
I am seeing on a whole bunch of PC Engine posts that the APU2 is supporting 1gb/s lines at full speed, some other posts report not.
Those reporting success seems to be running Linux (mostly debian) and with their own setup. Suggests the hardware is capable, but then it becomes a combination of the hardware and OS/routing/firewall platform.
Are there any more tuning items I should be looking at?
Hello @phillipmcmahon, I think all ok with your configuration.
Do you have a chance again to provide top output and press 1 with previously applied tuning?
Did you test bandwidth through WG tunnels?
I took these on the morning I applied the settings. Have not had chance yet to test the WG tunnels. I can do that if you think it will add value. Let me know.