TCP MSS does not applied on out-going packets

I am a newcomer to VyOS.

I am using DS-Lite for IPv4 communication, but some sites, such as github.com, give me errors.

From the tcpdump it looks like a tcp mss issue.
I have therefore put in a setting to set mss to 1400, but the packets coming from the server are only 1400, but the packets being sent are not.

I have also tried set interfaces {tunnel,ethernet} * ip adjust-mss clamp-mss-to-pmtu with no success.

This is my version and config.

vyos@vyos1# run show version
Version:          VyOS 1.4-rolling-202306190317
Release train:    current
vyos@vyos1# show
 firewall {
     interface eth0 {
         in {
             name LAN_IN
         }
         local {
             name LAN_LOCAL
         }
     }
     interface eth1 {
         in {
             ipv6-name WAN_IN
             name WAN_IN
         }
         local {
             ipv6-name WAN_LOCAL
             name WAN_LOCAL
         }
     }
     ipv6-name WAN_IN {
         default-action reject
         rule 10 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
     }
     ipv6-name WAN_LOCAL {
         default-action reject
         rule 10 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
         rule 20 {
             action accept
             protocol icmpv6
         }
         rule 30 {
             action accept
             destination {
                 port 546
             }
             protocol udp
             source {
                 port 547
             }
         }
         rule 40 {
             action accept
             destination {
                 port 33434-33534
             }
             protocol udp
         }
     }
     name LAN_IN {
         default-action accept
     }
     name LAN_LOCAL {
         default-action accept
     }
     name WAN_IN {
         default-action reject
         rule 10 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
     }
     name WAN_LOCAL {
         default-action reject
         rule 10 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
         rule 20 {
             action accept
             protocol icmp
         }
         rule 30 {
             action accept
             destination {
                 port 13231
             }
             protocol udp
         }
     }
 }
 interfaces {
     ethernet eth0 {
         address 192.168.1.2/24
         description LAN
         hw-id *****************
         ip {
             adjust-mss 1400
         }
         ipv6 {
             adjust-mss 1400
         }
     }
     ethernet eth1 {
         address *****************
         description WAN
         dhcpv6-options {
             duid *****************
             parameters-only
             pd 0 {
                 interface eth1 {
                     address 0
                 }
                 length 64
             }
             rapid-commit
         }
         hw-id *****************
         ip {
             adjust-mss 1400
         }
         ipv6 {
             address {
                 autoconf
             }
             adjust-mss 1400
         }
     }
     tunnel tun0 {
         address *****************
         encapsulation ipip6
         ip {
             adjust-mss 1400
         }
         ipv6 {
             adjust-mss 1400
         }
         remote *****************
         source-address *****************
     }
 }
 nat {
     source {
         rule 999 {
             outbound-interface tun0
             source {
                 address 192.168.0.0/16
             }
             translation {
                 address masquerade
             }
         }
     }
 }
 policy {
     route adjust-mss {
         interface eth0
         interface eth1
         interface tun0
         rule 10 {
             protocol tcp
             set {
                 tcp-mss 1400
             }
             tcp {
                 flags {
                     syn
                 }
                 mss 1401-16384
             }
         }
     }
     route6 adjust-mss {
         interface eth0
         interface eth1
         interface tun0
         rule 10 {
             protocol tcp
             tcp {
                 flags {
                     syn
                 }
                 mss 1400
             }
         }
     }
 }
 protocols {
     static {
         route 0.0.0.0/0 {
             interface tun0 {
                 distance 1
             }
         }
     }
 }
 service {
     dhcp-server {
         listen-address 192.168.1.2
         shared-network-name LAN {
             authoritative
             subnet 192.168.1.0/24 {
                 default-router 192.168.1.2
                 domain-name *****************
                 domain-search *****************
                 lease 86400
                 name-server 192.168.1.2
                 range 0 {
                     start 192.168.1.10
                     stop 192.168.1.99
                 }
             }
         }
     }
     dns {
         forwarding {
             allow-from 192.168.0.0/16
             cache-size 1000
             dhcp eth0
             listen-address 192.168.1.1
             listen-address 192.168.1.2
             name-server 2404:1a8:7f01:a::3 {
             }
             name-server 2404:1a8:7f01:b::3 {
             }
         }
     }
     ntp {
         allow-client {
             address ::/0
             address 192.168.0.0/16
         }
         server ntp.nict.jp {
         }
     }
     ssh {
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     conntrack {
         modules {
             ftp
             h323
             nfs
             pptp
             sip
             sqlnet
             tftp
         }
     }
     console {
         device ttyS0 {
             speed 115200
         }
     }
     host-name vyos1
     ipv6 {
         strict-dad
     }
     login {
         user vyos {
             authentication {
                 encrypted-password ***********************
                 plaintext-password ""
             }
         }
     }
     name-server 2404:1a8:7f01:a::3
     name-server 2404:1a8:7f01:b::3
     syslog {
         global {
             facility all {
                 level info
             }
             facility local7 {
                 level debug
             }
         }
     }
     time-zone Asia/Tokyo
 }

I wouldn’t clamp MSS on the WAN link or on any of the other v6 interfaces, that’s going to cause you to potentially drop packets sending stuff to the tunnel. (Look at ICMP messages for MSS-exceeded)

The v4 going through the tunnel should be clamped. What did your provider specify for default and/or max MTU for the tunnel to the AFTR? Set your MSS to MTU-40

Provided you aren’t blocking any other ICMP messages in the firewall then the stack should be honoring pmtu discovery on initial handshakes or throughout the session.

Looks like default discovery options are halfway there:

vyos@vyos:~$ sudo cat /proc/sys/net/ipv4/tcp_mtu_probing
0        # Should be 1 to properly use RFC4821
vyos@vyos:~$ sudo cat /proc/sys/net/ipv4/tcp_base_mss
1024     # Great!

What does ip link show tun0 give you? — Disclaimer: My tunnel is not for IPIP6

vyos@vyos:~$ ip link show tun0
4: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 53049 qdisc pfifo_fast state UNKNOWN mode DEFAULT group default qlen 500
    link/none
    alias Yggdrasil :)
1 Like

Note your WAN_IN and WAN_Local ipv4 firewall rules should be assigned to the interface having the IPv4 WAN address. Which is tun0 , not eth1 !
You probably don’t need mss-clamp for ipv6, but if you do enable it, use interfaces eth0 and eth1.
mss-clamp for ipv4 should go on eth0 and tun0

2 Likes

Thank you for replies.

Looks like default discovery options are halfway there:

I use set system sysctl parameter net.ipv4.tcp_mtu_probing value '1' and still can’t see github.com.

What does ip link show tun0 give you? — Disclaimer: My tunnel is not for IPIP6

The results of the run were as follows.

vyos@vyos1# ip link show tun0
5: tun0@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1476 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/tunnel6 <ipv6 address> peer <AFTR address> permaddr ca27:d656:c3be::

Note your WAN_IN and WAN_Local ipv4 firewall rules should be assigned to the interface having the IPv4 WAN address. Which is tun0 , not eth1 !

Oh, sorry.
I applied it to tun0.

You probably don’t need mss-clamp for ipv6, but if you do enable it, use interfaces eth0 and eth1.
mss-clamp for ipv4 should go on eth0 and tun0

Thank you. I removed it.

I found this comment and enabling offload gso,gro solved this issue.

vyos@vyos1# show interfaces ethernet eth0 offload
 gro
 gso

Thank you so much to everyone who commented.

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.