Low speed with NAT

I’m using vyos 1.3.2 on proxmox ve 7.1. There is 3 VLANs in the network. Everything works fine but the network speed. My broadband is 1Gbps with pppoe. The speedtest on vyos vm is 1Gbps, but other device only runs on 200-300Mbps. Include other vm running on the same proxmox ve server. Is there any solution?

That could be a large number of things, i do remember encountering an issue myself with poor nat performance that was caused by having lro offload enabled on the interfaces being nated. If you can provide your configs and the specs for the VMs/Hypervisor people may be able to give you more info.

My proxmox ve using E3-1220L v2 with 10GB DDR3 ECC memory, two BCM5716 and one MT26448 Ethernet cards. I bridge one 1Gb Ethernet card and the 10Gb card on proxmox ve as local network interface. Here is my vyos config

vyos@vyos.mi.lab# show
 firewall {
     all-ping enable
     broadcast-ping disable
     config-trap disable
     ipv6-receive-redirects disable
     ipv6-src-route disable
     ip-src-route disable
     log-martians enable
     name OUTSIDE-IN {
         default-action drop
         rule 10 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
     }
     name OUTSIDE-LOCAL {
         default-action drop
         rule 10 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
         rule 20 {
             action accept
             icmp {
                 type-name echo-request
             }
             protocol icmp
             state {
                 new enable
             }
         }
     }
     receive-redirects disable
     send-redirects enable
     source-validation disable
     syn-cookies enable
     twa-hazards-protection disable
 }
 interfaces {
     ethernet eth0 {
         description WAN
     }
     ethernet eth1 {
         vif 5 {
             address 10.5.255.254/16
             address 10.6.255.254/16
             description LAN
         }
         vif 20 {
             address 172.16.0.254/24
             description MGMT
         }
         vif 30 {
             address 10.1.255.254/16
             description GUEST
         }
     }
     loopback lo {
     }
     pppoe pppoe0 {
         authentication {
             password xxxxxx
             user xxxxxxx
         }
         default-route auto
         description WAN
         dhcpv6-options {
             pd 0 {
                 interface eth1 {
                     address 100
                 }
             }
         }
         ipv6 {
             address {
                 autoconf
             }
         }
         source-interface eth0
     }
 }
 nat {
     destination {
         rule 100 {
             description n1
             destination {
                 port 2222
             }
             inbound-interface pppoe0
             protocol tcp
             translation {
                 address 10.5.13.21
                 port 22
             }
         }
         rule 200 {
             description Black_PVE
             destination {
                 port 49000
             }
             inbound-interface pppoe0
             protocol tcp_udp
             translation {
                 address 10.5.222.0
                 port 8006
             }
         }
         rule 201 {
             description Black_port1
             destination {
                 port 49001
             }
             inbound-interface pppoe0
             protocol tcp_udp
             translation {
                 address 10.5.222.2
                 port 37192
             }
         }
         rule 202 {
             description Black_port2
             destination {
                 port 49002
             }
             inbound-interface pppoe0
             protocol tcp_udp
             translation {
                 address 10.5.222.2
                 port 38080
             }
         }
         rule 203 {
             description Black_rdp
             destination {
                 port 49003
             }
             inbound-interface pppoe0
             protocol tcp_udp
             translation {
                 address 10.5.222.1
                 port 3389
             }
         }
         rule 250 {
             description qyz_winserver
             destination {
                 port 10400
             }
             inbound-interface pppoe0
             protocol tcp_udp
             translation {
                 address 10.6.147.188
                 port 3389
             }
         }
         rule 251 {
             description qyz_winserver_ports
             destination {
                 port 10401-10499
             }
             inbound-interface pppoe0
             protocol tcp_udp
             translation {
                 address 10.6.147.188
                 port 10401-10499
             }
         }
     }
     source {
         rule 100 {
             outbound-interface pppoe0
             translation {
                 address masquerade
             }
         }
     }
 }
 service {
     dhcp-server {
         shared-network-name GUEST_DHCP {
             subnet 10.1.0.0/16 {
                 default-router 10.1.255.254
                 domain-name xxxx
                 lease 86400
                 name-server 10.1.255.254
                 range 0 {
                     start 10.1.0.10
                     stop 10.1.255.200
                 }
             }
         }
         shared-network-name LAN_DHCP {
             subnet 10.6.0.0/16 {
                 default-router 10.6.255.254
                 domain-name xxxx
                 lease 86400
                 name-server 10.6.255.254
                 range pool {
                     start 10.6.0.10
                     stop 10.6.255.250
                 }
             }
         }
         shared-network-name MGMT_DHCP {
             subnet 172.16.0.0/24 {
                 default-router 172.16.0.254
                 domain-name xxxx
                 lease 86400
                 name-server 172.16.0.254
                 range 0 {
                     start 172.16.0.10
                     stop 172.16.0.250
                 }
             }
         }
     }
     dns {
         dynamic {
             interface pppoe0 {
                 service cloudflare {
                     host-name xxxxxx
                     login xxxxxx
                     password xxxxxxx
                     protocol cloudflare
                     zone xxxxxxx
                 }
             }
         }
         forwarding {
             allow-from 0.0.0.0/32
             allow-from 0.0.0.0/0
             cache-size 0
             listen-address 0.0.0.0
         }
     }
     router-advert {
     }
     ssh {
         disable-password-authentication
         port 22
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     conntrack {
         modules {
             ftp
             h323
             nfs
             pptp
             sip
             sqlnet
             tftp
         }
     }
     console {
         device ttyS0 {
             speed 115200
         }
     }
     host-name vyos.mi.lab
     login {
         user vyos {
             authentication {
                 encrypted-password xxxxxxx
                 plaintext-password ""
                 public-keys vyos {
                     key xxxxxxx
                     type ssh-rsa
                 }
             }
         }
     }
     ntp {
         server time1.vyos.net {
         }
         server time2.vyos.net {
         }
         server time3.vyos.net {
         }
     }
     syslog {
         global {
             facility all {
                 level info
             }
             facility protocols {
                 level debug
             }
         }
     }
     time-zone Asia/Shanghai
 }

This is my switch config

#
#switch-type "Blade Network Technologies Rack Switch G8124"
#Software Version 1.1.1.5
#!!!!DO NOT EDIT ANYTHING ABOVE THIS LINE!!!!
#
!
!
!
spanning-tree stp 1 vlan 5
interface port 1
tagging
tag-pvid
pvid 5
!
interface port 2
pvid 5
!
interface port 3
pvid 5
!
interface port 4
pvid 5
!
interface port 5
pvid 5
!
interface port 6
pvid 5
!
interface port 7
pvid 5
!
interface port 8
pvid 5
!
portchannel 1 member 1
portchannel 1 enable
!

vlan 5
enable
name "VLAN5"
member 1-8
!
!

end
1 Like

By the way, i have tried using iperf3 to test the bandwidth. There is weird problem. When i running ‘iperf -c xx.xx.xx.xx’ from the device connected to vyos router, the speed is almost 8Gbps. But when i running ‘iperf -c xx.xx.xx.xx -R’ from that device, the speed only is 3Gbps. The speed between proxmox ve host is 1~3Gbps, but the speed between the other vm on proxmox ve are both 8Gbps. This is very weird, i don’t know which part goes wrong.

Try to enable offloads

set interfaces ethernet ethX offload gso
set interfaces ethernet ethX offload lro
set interfaces ethernet ethX offload sg
set interfaces ethernet ethX offload tso

Thanks, the bandwidth problem is fixed. but it’s not work for NAT speed.

Try to delete firewall and check if you don’t have tcp-mss issue.