Strange browsing issues through VyOS Router

General questions
1

Hello, I’m looking for a fresh pair of eyes to validate my VyOS config. The issue i’m experiencing is to do with web browsing. I’m able to connect to and view most web pages however some just refuse to load.

I’ve tried browsing to the same pages in multiple browsers (Firefox / Chrome / IE) all of which display the same behavior. The pages appear to get stuck doing a DNS lookup, here are a couple of examples:

waiting cdn5.nflximg.net
waiting for platform.twitter.com
waiting for wiki.centos.com

It doesn’t seem to make any difference if the request is made using HTTP or HTTPS the problem remains. If I take the domain ame and do a manual nslookup the domain is resolved successfully.

It also doesn’t seem to make any difference if I point my PC at the router for DNS or use Google’s DNS (8.8.8.8 / 8.8.4.4) directly in the PC’s network config.

My ISP is Plusnet and I have an FTTC connection, I’m able to download ISO’s through the router at ~8MB/s again proving the connection is up and stable.

I have VyOS running on an Intel Atom Mini ITX PC (D945GCLF2). I’ve added a second PCI NIC to this setup (Intel Corporation 82541PI Gigabit Ethernet Controller). The board has a 1GB stick of RAM at present

Currently the on-board Realtek NIC (ETH1) is connected to the BT Open Reach supplied FTTC modem. The add-on NIC (Intel) is connected to my switch which is then connected to my PC.

I’m looking for pointers really to debug this issue further. As I said I’m able to browse the internet through this router. I’m writing this post on the same connection. Is the issue likely to be a resource allocation problem? Hash table size setting? Or have I made a configuration mistake?

I have tried experimenting with various tcp-mss values but this has not made any difference. I have now removed the policy statement from this configuration.

The config I have at present is as follows:

firewall {
    all-ping enable
    broadcast-ping disable
    config-trap disable
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians enable
    name eth0-in {
        default-action accept
        description "Wied network to other networks."
    }
    name eth0-local {
        default-action accept
        description "Wied network to router."
    }
    name pppoe-in {
        default-action drop
        description "Internet to internal networks"
        rule 1 {
            action accept
            description "Allow established/related"
            log disable
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            description "Drop invalid state"
            log enable
            state {
                invalid enable
            }
        }
    }
    name pppoe-local {
        default-action drop
        description "Internet to router"
        rule 1 {
            action accept
            description "Allow established/related"
            log disable
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            description "Drop invalidstate"
            log enable
            state {
                invalid enable
            }
        }
    }
    receive-redirects disable
    send-redirects enable
    source-validation disable
    syn-cookies enable
}
interfaces {
    ethernet eth0 {
        address 192.168.1.252/24
        duplex auto
        firewall {
            in {
                name eth0-in
            }
            local {
               name eth0-local
            }
        }
        hw-id 00:1b:21:80:58:0b
        smp_affinity auto
        speed auto
    }
    ethernet eth1 {
        duplex auto
        hw-id 00:1c:c0:f1:d2:c5
        pppoe 0 {
            default-route auto
            firewall {
                in {
                    name pppoe-in
                }
                local {
                    name pppoe-local
                }
            }
            mtu 1492
            name-server auto
            password ****************
            user-id ***USERNAME HERE***
        }
        smp_affinity auto
        speed auto
    }
    loopback lo {
    }
}
nat {
    source {
        rule 1024 {
            log disable
            outbound-interface pppoe0
            protocol all
            source {
                address 192.168.1.0/24
            }
            translation {
                address masquerade
            }
        }
    }
}
service {
    dns {
        forwarding {
            cache-size 0
            listen-on eth0
            name-server 8.8.8.8
            name-server 8.8.4.4
            system
        }
    }
    ssh {
        port 22
    }
}
system {
    config-management {
        commit-revisions 20
    }
    console {
        device ttyS0 {
            speed 9600
        }
    }
    host-name vyos
    login {
        user vyos {
            authentication {
                encrypted-password ****************
            }
            level admin
        }
    }
    name-server 8.8.8.8
    ntp {
        server 0.pool.ntp.org {
        }
        server 1.pool.ntp.org {
        }
        server 2.pool.ntp.org {
        }
    }
    package {
        auto-sync 1
        repository community {
            components main
            distribution hydrogen
            password ****************
            url http://packages.vyos.net/vyos
            username ""
        }
        repository squeeze {
            components "main contrib non-free"
            distribution squeeze
            password ****************
            url http://mirrors.kernel.org/debian
            username ""
        }
    }
    syslog {
        global {
            facility all {
                level notice
            }
            facility protocols {
                level debug
            }
        }
    }
    time-zone UTC
}

This is the MSS clamping config I was using:

 route wan-out {
     rule 10 {
         protocol tcp
         set {
             tcp-mss 1200
         }
         tcp {
             flags SYN,RST
         }
     }
 }

Applied to the pppoe0 interface here:

 ethernet eth1 {
        duplex auto
        hw-id 00:1c:c0:f1:d2:c5
        pppoe 0 {
            default-route auto
            firewall {
                in {
                    name pppoe-in
                }
                local {
                    name pppoe-local
                }
            }
            mtu 1492
            name-server none
            password ****************
            policy {
                route wan-out
            }
            user-id <<<USERNAME>>>
        }
        smp_affinity auto
        speed auto
    }

This method appears to have changed from the method used in previous versions of Vyatta, see below:


modify pppoe-out {
        description "TCP clamping"
        rule 1 {
            action modify
            modify {
                tcp-mss 1452
            }
            protocol tcp
            tcp {
                flags SYN
            }
        }
    }

Packets appear to being dropped by the policy settings:

IPv4 Policy Route "wan-out":

 Active on (pppoe0,ROUTE)

rule  action   proto     packets  bytes
----  ------   -----     -------  -----
10    set      tcp       0        0
  condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0 tcpflags: 0x06/0x06 TCPMSS set 500

10000 drop     all       10183    4814029
  condition - saddr 0.0.0.0/0 daddr 0.0.0.0/0

Interface info:

vyos@vyos:~$ show interfaces ethernet detail
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1b:21:80:58:0b brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.252/24 brd 192.168.1.255 scope global eth0
    inet6 fe80::21b:21ff:fe80:580b/64 scope link
       valid_lft forever preferred_lft forever

    RX:  bytes    packets     errors    dropped    overrun      mcast
       1744432      12312          0          0          0          8
    TX:  bytes    packets     errors    dropped    carrier collisions
       5150418      12150          0          0          0          0
eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1c:c0:f1:d2:c5 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21c:c0ff:fef1:d2c5/64 scope link
       valid_lft forever preferred_lft forever

    RX:  bytes    packets     errors    dropped    overrun      mcast
       5065491      10432          0          0          0          0
    TX:  bytes    packets     errors    dropped    carrier collisions
       1561322       9594          0          0          0          0

vyos@vyos:~$ show interfaces pppoe
pppoe0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc pfifo_fast state UNKNOWN qlen 100
    link/ppp
    inet xxx.xxx.xxx.xxx peer 195.166.130.152/32 scope global pppoe0

    RX:  bytes    packets     errors    dropped    overrun      mcast
       4847388      10380          0          0          0          0
    TX:  bytes    packets     errors    dropped    carrier collisions
       1365118       9567          0          0          0          0
2

After doing some more reading and a bit more digging I checked the iptables rules using:

On my VyOS router there wasn’t anything in there for MSS clamping with the policy referenced above applied.

Issuing the command below appears to have corrected the issue I was facing, I’ll do some more testing tomorrow to confirm this.

Am I missing something or should it be possible to set this without manually hacking the iptables rules. This configuration has been referenced multiple times on the web:

 modify pppoe-out {
        description "TCP clamping"
        rule 1 {
            action modify
            modify {
                tcp-mss 1452
            }
            protocol tcp
            tcp {
                flags SYN
            }
        }
    }

However “modify” does not appear to be an available action:

Possible completions:
   drop         Rule action to drop
   reject       Rule action to reject
   accept       Rule action to accept
   inspect      Rule action to inspect

My next issue will be getting this iptables hack to stick around after a reboot.

It would appear that adding the iptables rule to this file does the job:

/config/scripts/vyatta-postconfig-bootup.script

Now if another user could confirm whether this is expected behavior of the firewall settings to reject the modify option and in addition to this whether the work around documented above is the only option for this I’d be very grateful.

3

Hi,

I also had this problem and got round it by inserting this:

sudo iptables -t mangle -A POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o pppoe0 -j TCPMSS --clamp-mss-to-pmtu

into vyatta-postconfig-bootup.script

Hope it helps

SteveP

4

Hi Steve,

Thanks for letting me know, its good to know that I’m not the only one to have run into this issue. Thanks for sharing your iptables rule as well!

Regards,

Chris

5

Does this work?

set policy route pppoe-out rule 1 set tcp-mss pmtu
6

this also fixed it for me! where do i find the vyatta-postconfig-bootup.script file?
The bug seems to be still there in 2017!

Thanks for the workaround…

7

Does this not work?

set policy route pppoe-out rule 1 protocol 'tcp'
set policy route pppoe-out rule 1 set tcp-mss 'pmtu'
set policy route pppoe-out rule 1 tcp flags 'SYN'

set interfaces ... policy route pppoe-out
8

Hi,

It does not … with ‘commit’ i get:

[code][ interfaces ethernet eth0 pppoe 0 policy route PPPOE-OUT ]
iptables: Invalid argument. Run `dmesg’ for more information.

[[interfaces ethernet eth0]] failed
Commit failed
[/code]

dmesg returns:

Thanks for your help!

9

i did upgrade to VyOS 1.1.8 and can confirm that the problem still exists there…

10

Ah, and it is also in the nightly 1.2 version.

See ⚓ T490 policy route path-MTU clamping

11

great thanks… while the workaround is working it would of course be perfect to have this bug fixed :slight_smile: