Vpp availability

So I work for an isp and we were curious on how vyos would work as nat-servers. We sadly ran into I brick wall and want to try out the vpp to se if we could get better performance by using that. The rolling release referenced in vpp documents however are not available, any know where I can get my hands on it?

Thanks in advance

as of now, you can’t
Work in progress to integrate VPP in rolling so you may want to follow blog posts
We will post once it’s in rolling

2 Likes

Thank you for the instant response, that very sad to hear.

Maybe someone got that iso saved somewhere :slight_smile:

Or assistance in optimizing our setup perhaps, we tried many different hardware, ranging from legacy to fairly new. Always with x710 nics

We plateau at about a /21 on the inside and conntrack toping at 80k or there about, anyone got tips?

Regarding x710 I assume you already seen this?

Along with the usual suspects to try out different offloading options under interface settings in VyOS, note that not all offloading options turns out to be healthy so recommended to try one at a time.

Then to tweak conntrack you can adjust several settings in VyOS.

For example hash-size, table-size and expect-table-size but also the timers to make them as small as possible (otherwise many defaults in Linux kernel are like 2 weeks for established TTL which is a bit too much to waste RAM on).

Common things to check/evaluate (note that not all nic offloading options are healthy depending on hardware and drivers):

firewall {
    global-options {
        all-ping "enable"
        broadcast-ping "disable"
        ip-src-route "disable"
        ipv6-receive-redirects "disable"
        ipv6-source-validation "strict"
        ipv6-src-route "disable"
        log-martians "enable"
        receive-redirects "disable"
        resolver-cache
        resolver-interval "60"
        send-redirects "disable"
        source-validation "strict"
        syn-cookies "enable"
        twa-hazards-protection "disable"
    }
...

interfaces {
    ethernet ethX {
        ip {
            arp-cache-timeout "240"
        }
        offload {
            gro
            gso
            lro
            rfs
            rps
            sg
            tso
        }
        ring-buffer {
            rx "4096"
            tx "4096"
        }
...

system {
    conntrack {
        expect-table-size "10485760"
        hash-size "10485760"
        log {
            icmp {
                new
            }
            other {
                new
            }
            tcp {
                new
            }
            udp {
                new
            }
        }
        table-size "10485760"
        timeout {
            icmp "10"
            other "600"
            tcp {
                close "10"
                close-wait "30"
                established "600"
                fin-wait "30"
                last-ack "30"
                syn-recv "30"
                syn-sent "30"
                time-wait "30"
            }
            udp {
                other "600"
                stream "600"
            }
        }
    }
...

    ip {
        arp {
            table-size "32768"
        }
        disable-directed-broadcast
        multipath {
            layer4-hashing
        }
    }
    ipv6 {
        disable-forwarding
        multipath {
            layer4-hashing
        }
        neighbor {
            table-size "32768"
        }
    }
...

    option {
        ctrl-alt-delete "reboot"
        http-client {
            source-interface "ethX"
        }
        keyboard-layout "se-latin1"
        performance "throughput"
        reboot-on-panic
        root-partition-auto-resize
        ssh-client {
            source-interface "ethX"
        }
        startup-beep
        time-format "24-hour"
    }
    sysctl {
        parameter kernel.core_uses_pid {
            value "1"
        }
        parameter kernel.sysrq {
            value "0"
        }
        parameter vm.swappiness {
            value "1"
        }
        parameter vm.vfs_cache_pressure {
            value "50"
        }
    }
...

As I recall it its about 300 bytes or so per conntrack entry incl everything.

So the 1M in my example would at most eat up 300MB + 300MB = 600MB for the conntrack. The expect table is for related traffic like you have echo-request in one direction and wish to allow for echo-reply to return.

A general recommendation (if you got the RAM for this) is to have a 1:1 relation between hash-table and table-size to limit amount of CPU cycles needed to locate an entry.

That is setting something like 100M for hash-size+table-size and 1M for expect-table-size should mean a peak of about 28.6GB of RAM in total for the conntrack when filled to max. So a box with 32GB RAM should work.

If you got more RAM to spare then you can bump the hash-table+table-size even further.

You misunderstood the message. VPP will be in all rollings very soon. :slight_smile:

3 Likes

re: … I work for an isp … & … how vyos would work as nat-servers …
I use some VyOS virtual routers at my ISP to handle cgnat to all of my fiber and wireless customers ( my customers who do not have/use live IP addresses ).

  • My VyOS cgnnat router #1 to my wireless customers during busy peak hours runs up to about 1.4-Gig+ every day.
  • My VyOS cgnnat router #2 to my fiber/GPON customers during busy peak hours runs up to about 2.7-Gig+ every day.
    I have about 1k wireless customers and about 1k fiber/GPON customers.

So yea , VyOS cgnnat does work well and I am very happy with it.
I’ve tested other cgn nat routers such as the Mikrotik CHR , but with this router things were slower and I had some complaints. Since switching to VyOS cgnnat ( about a year ago ) , all of those old complaints and problems have gone away.

In my cgnnat , I breakup a single live IP address into an entire 100.64.x.x/24 network – where I use 250 ports from the live IP address to each customer nat IP.
– A little math here —
A single live IP address has 65,535 ports , if you use 250 port groups to nat customers , then you can have ( 65,535 divided-by 250 = 262.14 ) easily nat a /24 to your customers.
In my case , I have two Vyos cgnnat routers , each router is using 8 live IP address. Each cgnnat router can handle a /21 of nat to customers.

  • a /21 has 2,048 IP addresses
    Note; I have 12 Proxmox servers. All of my servers ( including VyOS routers ) are virtual machines.

North Idaho Tom Jones

CGNAT is a cool feature but not related to VPP in this case.