PPPoE WAN working on router but not LAN clients

trombone · October 13, 2024, 8:59pm

I’m struggling to get a PPPoE internet connection working on my (virtualized) VyOS router. I can get the router to have functioning Internet, but the LAN clients never have better than partial internet (some websites work, others don’t). I think it has to do with MTU/MRU/MSS settings, but not sure.

My provider (source) provides these settings:

PPPoE via VLAN 6 (802.1q).
PPPoE authentication PAP with username and password (e.g. internet / internet).
Maximum packet size (MTU) 1500 bytes (rfc4638)
Obtain IPv4 address + DNS servers via PPPoE
Obtain IPv6 address range + DNS servers (IPv6) via DHCPv6-PD request (in PPPoE). Use one address for the router from this range.

which I implemented as follows (based on here, here, here):

set interfaces pppoe pppoe0 description 'KPN WAN'
set interfaces pppoe pppoe0 authentication username 'internet'
set interfaces pppoe pppoe0 authentication password 'internet'
set interfaces pppoe pppoe0 source-interface 'eth1.6'
# Ensure we get DNS from ISP, i.e. no-peer-dns must be unset
delete interfaces pppoe pppoe0 no-peer-dns

After fiddling with MTU/MSS settings a bit, these settings give only partial internet (e.g. google.com works, duckduckgo.com does not, unpredictable which website works, ping always):

set interfaces ethernet eth1 mtu '1512'
set interfaces ethernet eth1 vif 6 mtu '1508'
set interfaces pppoe pppoe0 mtu '1500'
set interfaces ethernet eth1 vif 6 ip adjust-mss '1460'

Lowering MTU fixes internet on the router only, I can wget any website, but clients connected to LAN still have partial internet as the above situation:

set interfaces ethernet eth1 mtu '1412'
set interfaces ethernet eth1 vif 6 mtu '1408'
set interfaces pppoe pppoe0 mtu '1400'
set interfaces ethernet eth1 vif 6 ip adjust-mss '1360'

With the pppoe MTU 1400 settings (which works on my router), I capture the following on my LAN client when visiting google.com, which sets the frame size to 1400:

Duckduckgo doesn’t respond (=packets get lost?)

Rest of config looks like:

set firewall all-ping 'enable'
set firewall broadcast-ping 'disable'
set firewall ip-src-route 'disable'
set firewall ipv6-receive-redirects 'disable'
set firewall ipv6-src-route 'disable'
set firewall log-martians 'enable'
set firewall name FW_2LOCAL default-action 'drop'
set firewall name FW_2LOCAL rule 200 action 'accept'
set firewall name FW_2LOCAL rule 200 description 'accept established/related'
set firewall name FW_2LOCAL rule 200 log 'disable'
set firewall name FW_2LOCAL rule 200 state established 'enable'
set firewall name FW_2LOCAL rule 200 state related 'enable'
set firewall name FW_2LOCAL rule 210 action 'accept'
set firewall name FW_2LOCAL rule 210 description 'accept dhcp'
set firewall name FW_2LOCAL rule 210 destination port '67-68'
set firewall name FW_2LOCAL rule 210 log 'disable'
set firewall name FW_2LOCAL rule 210 protocol 'udp'
set firewall name FW_2LOCAL rule 220 action 'accept'
set firewall name FW_2LOCAL rule 220 description 'accept dns'
set firewall name FW_2LOCAL rule 220 destination port '53'
set firewall name FW_2LOCAL rule 220 log 'disable'
set firewall name FW_2LOCAL rule 220 protocol 'udp'
set firewall name FW_2LOCAL rule 230 action 'accept'
set firewall name FW_2LOCAL rule 230 description 'accept ssh'
set firewall name FW_2LOCAL rule 230 destination port '22'
set firewall name FW_2LOCAL rule 230 log 'disable'
set firewall name FW_2LOCAL rule 230 protocol 'tcp'
set firewall name FW_ACCEPT default-action 'accept'
set firewall name FW_ACCEPT rule 200 action 'drop'
set firewall name FW_ACCEPT rule 200 description 'drop invalid'
set firewall name FW_ACCEPT rule 200 state invalid 'enable'
set firewall name FW_DROP default-action 'drop'
set firewall name FW_GUEST2INFRA default-action 'drop'
set firewall name FW_GUEST2INFRA rule 200 action 'accept'
set firewall name FW_GUEST2INFRA rule 200 description 'accept established/related'
set firewall name FW_GUEST2INFRA rule 200 log 'disable'
set firewall name FW_GUEST2INFRA rule 200 state established 'enable'
set firewall name FW_GUEST2INFRA rule 200 state related 'enable'
set firewall name FW_GUEST2TRUST default-action 'drop'
set firewall name FW_GUEST2TRUST rule 200 action 'accept'
set firewall name FW_GUEST2TRUST rule 200 description 'accept established/related'
set firewall name FW_GUEST2TRUST rule 200 log 'disable'
set firewall name FW_GUEST2TRUST rule 200 state established 'enable'
set firewall name FW_GUEST2TRUST rule 200 state related 'enable'
set firewall name FW_IOT2INFRA default-action 'drop'
set firewall name FW_IOT2INFRA rule 200 action 'accept'
set firewall name FW_IOT2INFRA rule 200 description 'accept established/related'
set firewall name FW_IOT2INFRA rule 200 log 'disable'
set firewall name FW_IOT2INFRA rule 200 state established 'enable'
set firewall name FW_IOT2INFRA rule 200 state related 'enable'
set firewall name FW_IOT2WAN default-action 'drop'
set firewall name FW_IOT2WAN rule 200 action 'accept'
set firewall name FW_IOT2WAN rule 200 description 'accept established/related'
set firewall name FW_IOT2WAN rule 200 log 'disable'
set firewall name FW_IOT2WAN rule 200 state established 'enable'
set firewall name FW_IOT2WAN rule 200 state related 'enable'
set firewall name FW_TRUST2INFRA default-action 'drop'
set firewall name FW_TRUST2INFRA rule 200 action 'accept'
set firewall name FW_TRUST2INFRA rule 200 description 'accept established/related'
set firewall name FW_TRUST2INFRA rule 200 log 'disable'
set firewall name FW_TRUST2INFRA rule 200 state established 'enable'
set firewall name FW_TRUST2INFRA rule 200 state related 'enable'
set firewall name FW_WAN2ALL default-action 'drop'
set firewall name FW_WAN2ALL rule 200 action 'accept'
set firewall name FW_WAN2ALL rule 200 description 'accept established/related'
set firewall name FW_WAN2ALL rule 200 state established 'enable'
set firewall name FW_WAN2ALL rule 200 state related 'enable'
set firewall name FW_WAN2INFRA default-action 'drop'
set firewall name FW_WAN2INFRA rule 200 action 'accept'
set firewall name FW_WAN2INFRA rule 200 description 'accept established/related'
set firewall name FW_WAN2INFRA rule 200 state established 'enable'
set firewall name FW_WAN2INFRA rule 200 state related 'enable'
set firewall name FW_WAN2LOCAL default-action 'drop'
set firewall name FW_WAN2LOCAL enable-default-log
set firewall name FW_WAN2LOCAL rule 200 action 'accept'
set firewall name FW_WAN2LOCAL rule 200 description 'accept established/related'
set firewall name FW_WAN2LOCAL rule 200 log 'enable'
set firewall name FW_WAN2LOCAL rule 200 state established 'enable'
set firewall name FW_WAN2LOCAL rule 200 state related 'enable'
set firewall name FW_WAN2LOCAL rule 210 action 'accept'
set firewall name FW_WAN2LOCAL rule 210 description 'wireguard'
set firewall name FW_WAN2LOCAL rule 210 destination port '51820'
set firewall name FW_WAN2LOCAL rule 210 log 'enable'
set firewall name FW_WAN2LOCAL rule 210 protocol 'udp'
set firewall name FW_WAN2LOCAL rule 210 state new 'enable'
set firewall receive-redirects 'disable'
set firewall send-redirects 'enable'
set firewall source-validation 'disable'
set firewall syn-cookies 'enable'
set firewall zone GUEST default-action 'drop'
set firewall zone GUEST from INFRA firewall name 'FW_ACCEPT'
set firewall zone GUEST from IOT firewall name 'FW_DROP'
set firewall zone GUEST from LOCAL firewall name 'FW_ACCEPT'
set firewall zone GUEST from TRUSTED firewall name 'FW_ACCEPT'
set firewall zone GUEST from WAN firewall name 'FW_WAN2ALL'
set firewall zone GUEST interface 'br100.30'
set firewall zone INFRA default-action 'drop'
set firewall zone INFRA from GUEST firewall name 'FW_GUEST2INFRA'
set firewall zone INFRA from IOT firewall name 'FW_IOT2INFRA'
set firewall zone INFRA from LOCAL firewall name 'FW_ACCEPT'
set firewall zone INFRA from TRUSTED firewall name 'FW_TRUST2INFRA'
set firewall zone INFRA from WAN firewall name 'FW_WAN2INFRA'
set firewall zone INFRA interface 'br100.10'
set firewall zone IOT default-action 'drop'
set firewall zone IOT from GUEST firewall name 'FW_DROP'
set firewall zone IOT from INFRA firewall name 'FW_ACCEPT'
set firewall zone IOT from LOCAL firewall name 'FW_ACCEPT'
set firewall zone IOT from TRUSTED firewall name 'FW_ACCEPT'
set firewall zone IOT from WAN firewall name 'FW_WAN2ALL'
set firewall zone IOT interface 'br100.40'
set firewall zone LOCAL default-action 'drop'
set firewall zone LOCAL from GUEST firewall name 'FW_2LOCAL'
set firewall zone LOCAL from INFRA firewall name 'FW_ACCEPT'
set firewall zone LOCAL from IOT firewall name 'FW_2LOCAL'
set firewall zone LOCAL from TRUSTED firewall name 'FW_2LOCAL'
set firewall zone LOCAL from WAN firewall name 'FW_WAN2LOCAL'
set firewall zone LOCAL local-zone
set firewall zone TRUSTED default-action 'drop'
set firewall zone TRUSTED from GUEST firewall name 'FW_GUEST2TRUST'
set firewall zone TRUSTED from INFRA firewall name 'FW_ACCEPT'
set firewall zone TRUSTED from IOT firewall name 'FW_IOT2INFRA'
set firewall zone TRUSTED from LOCAL firewall name 'FW_ACCEPT'
set firewall zone TRUSTED from WAN firewall name 'FW_WAN2ALL'
set firewall zone TRUSTED interface 'br100.20'
set firewall zone TRUSTED interface 'wg0'
set firewall zone WAN default-action 'drop'
set firewall zone WAN from GUEST firewall name 'FW_ACCEPT'
set firewall zone WAN from INFRA firewall name 'FW_ACCEPT'
set firewall zone WAN from IOT firewall name 'FW_IOT2WAN'
set firewall zone WAN from LOCAL firewall name 'FW_ACCEPT'
set firewall zone WAN from TRUSTED firewall name 'FW_ACCEPT'
set firewall zone WAN interface 'pppoe0'
set interfaces bridge br100 enable-vlan
set interfaces bridge br100 member interface eth0 allowed-vlan '10'
set interfaces bridge br100 member interface eth0 allowed-vlan '20'
set interfaces bridge br100 member interface eth0 allowed-vlan '30'
set interfaces bridge br100 member interface eth0 allowed-vlan '40'
set interfaces bridge br100 stp
set interfaces bridge br100 vif 10 address '172.17.10.1/24'
set interfaces bridge br100 vif 10 description 'VLAN10-Mgmt'
set interfaces bridge br100 vif 20 address '172.17.20.1/24'
set interfaces bridge br100 vif 20 description 'VLAN20-Trusted'
set interfaces bridge br100 vif 30 address '172.17.30.1/24'
set interfaces bridge br100 vif 30 description 'VLAN30-Guest'
set interfaces bridge br100 vif 40 address '172.17.40.1/24'
set interfaces bridge br100 vif 40 description 'VLAN40-IoT'
set interfaces ethernet eth0 description 'LAN'
set interfaces ethernet eth1 mtu '1512'
set interfaces ethernet eth1 vif 6 description 'KPN WAN'
set interfaces ethernet eth1 vif 6 ip adjust-mss '1460'
set interfaces ethernet eth1 vif 6 mtu '1508'
set interfaces loopback lo
set interfaces pppoe pppoe0 authentication password 'internet'
set interfaces pppoe pppoe0 authentication username 'internet'
set interfaces pppoe pppoe0 description 'KPN WAN'
set interfaces pppoe pppoe0 mtu '1500'
set interfaces pppoe pppoe0 source-interface 'eth1.6'
set interfaces wireguard wg0 address '172.17.50.1/24'
set nat source rule 5001 description 'Exclude roadwarrior VPN'
set nat source rule 5001 destination address '172.17.50.0/24'
set nat source rule 5001 exclude
set nat source rule 5001 outbound-interface 'pppoe0'
set nat source rule 5001 protocol 'all'
set nat source rule 5001 translation address 'masquerade'
set nat source rule 5010 description 'Masquerade for WAN'
set nat source rule 5010 outbound-interface 'pppoe0'
set nat source rule 5010 protocol 'all'
set nat source rule 5010 source address '172.17.0.0/16'
set nat source rule 5010 translation address 'masquerade'
set protocols igmp-proxy
set service broadcast-relay
set service dhcp-server shared-network-name vlan10 authoritative
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 default-router '172.17.10.1'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 name-server '172.17.10.1'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 range vlan10range start '172.17.10.100'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 range vlan10range stop '172.17.10.254'
set service dhcp-server shared-network-name vlan20 authoritative
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 default-router '172.17.20.1'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 name-server '172.17.20.1'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 range vlan20range start '172.17.20.100'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 range vlan20range stop '172.17.20.254'
set service dhcp-server shared-network-name vlan30 authoritative
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 default-router '172.17.30.1'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 name-server '172.17.30.1'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 range vlan30range start '172.17.30.100'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 range vlan30range stop '172.17.30.254'
set service dhcp-server shared-network-name vlan40 authoritative
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 default-router '172.17.40.1'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 name-server '172.17.40.1'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 range vlan40range start '172.17.40.100'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 range vlan40range stop '172.17.40.254'
set service dns forwarding allow-from '172.17.0.0/16'
set service dns forwarding cache-size '100004'
set service dns forwarding dhcp 'pppoe0'
set service dns forwarding listen-address '172.17.10.1'
set service dns forwarding listen-address '172.17.20.1'
set service dns forwarding listen-address '172.17.30.1'
set service dns forwarding listen-address '172.17.40.1'
set service dns forwarding listen-address '172.17.50.1'
set service mdns repeater interface 'br100.20'
set service mdns repeater interface 'br100.30'
set service mdns repeater interface 'br100.10'
set system config-management commit-revisions '100'
set system conntrack modules ftp
set system conntrack modules h323
set system conntrack modules nfs
set system conntrack modules pptp
set system conntrack modules sip
set system conntrack modules sqlnet
set system conntrack modules tftp
set system console device ttyS0 speed '115200'
set system domain-name 'lan.vanwerkhoven.org'
set system host-name 'vyos'
set system name-server '172.17.10.1'
set system syslog global facility all level 'info'
set system syslog global facility local7 level 'debug'

VyOS version:

vyos@vyos:~$ show version
Version:          VyOS 1.4-rolling-202307120317
Release train:    current

Built by:         [email protected]
Built on:         Wed 12 Jul 2023 03:17 UTC
Build UUID:       481a5c2e-f2d1-4098-880b-c6bb959aaa36
Build commit ID:  5a81df95612424

I’ve rebooted both the router and client after updating the settings.

Any ideas why my LAN clients don’t have internet? What MTU/MSS/MRU settings do I need? E.g. are VLAN tags part of MTU or not? I see different approaches. Thanks in advance

marc_s · October 14, 2024, 2:18pm

Hoi!

Could you try the clamping setting on the pppoe0 interface, like so:

set interfaces pppoe pppoe0 ip adjust-mss 'clamp-mss-to-pmtu'

I think your first config has the correct MTUs: 1512 for eth1, 1508 (1512-4) for eth1 vif 6 and 1500 for pppoe0.

trombone · October 16, 2024, 7:05am

Goedemorgen,

I did try that previously in a less structured approach, but will check again. Just to be sure: MSS clamping is a ‘hack’ that fixes TCP but still leaves e.g. UDP at risk of dropped packets (datagrams), right? Can it be that I need MRU as well? I notice it’s not available in my VyOS version.

For reference, my (working) Mikrotik config on a similar provider (Telfort) a few years ago looked like below.

No MSS clamping
Explicit MRU at the same value as MTU (1500)

# Set SFP VLAN to 6 (KPN internet value)
/interface vlan set vlan-id=6 [find interface=sfp1]

# Disable DHCP (KPN uses PPPOE)
/ip dhcp-client disable [find interface=WAN_VLAN]

# Disable auto-negotiation
/interface ethernet set [ find default-name=sfp1 ] auto-negotiation=no arp=proxy-arp l2mtu=1598 

# Just to be sure
/routing bgp instance set default disabled=yes

# Add PPPoE, with or without IPv6
/ppp profile set *0                only-one=yes use-compression=yes use-upnp=no use-ipv6=no
/ppp profile add name=default-ipv6 only-one=yes use-compression=yes use-upnp=no use-ipv6=yes

/interface pppoe-client add add-default-route=yes allow=pap disabled=no interface=WAN_VLAN keepalive-timeout=20 max-mru=1500 max-mtu=1500 name=pppoe-client profile=default-ipv6 use-peer-dns=yes password=internet user=internet
/interface pppoe-client add add-default-route=yes allow=pap disabled=no interface=WAN_VLAN keepalive-timeout=20 max-mru=1500 max-mtu=1500 name=pppoe-kpn-client profile=default use-peer-dns=yes password=internet user=internet

# Set proxy-arp on bridge
/interface bridge set 0 arp=proxy-arp

fernando · October 16, 2024, 12:03pm

in VyOS is able to configure the MRU on pppoe interfaces, usually is the same value that the MTU:

vyos@vyos# set interfaces pppoe pppoe0 mru
Possible completions:
   <128-16384>          Maximum Receive Unit in byte

trombone · October 21, 2024, 7:32pm

Thanks, but unfortunately my version does not have this, it seem this was introduced the last 1-2 years, or am I doing something wrong?

vyos@vyos# set interfaces pppoe pppoe0 mru

  Configuration path: interfaces pppoe pppoe0 [mru] is not valid
      
[edit]

tjh · October 21, 2024, 8:33pm

I think your version is too old for the MRU patch, it was done in Oct of last year, your 1.4 rolling is from July.
I would suggest trying 1.5 rolling instead.

system · November 4, 2024, 8:34pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

tjh · November 10, 2024, 9:25pm

Opened at request of @trombone

trombone · November 10, 2024, 9:58pm

I think I solved it by trying again with apparently slightly different settings, and with some inspiration from @marc_s. The MTU settings I have are:

set interfaces pppoe pppoe0 mtu '1500'
set interfaces ethernet eth1 vif 6 ip adjust-mss clamp-mss-to-pmtu
set interfaces pppoe pppoe0 ip adjust-mss 1352

I think I this is not fully correct, 1352 seems to low, and I’m not sure if I need to clamp on both eth1.6 and pppoe. Only setting set interfaces pppoe pppoe0 ip adjust-mss clamp-mss-to-pmtu did not work. Some more permutations are needed once I have another slot to debug home internet

Apachez · November 11, 2024, 6:15pm

Also allowing ICMP to make path mtu discovery properly work would also help for the case of TCP-traffic (but UDP doesnt seem to grasp the good stuff that PMTU brings).

diedvdyk · November 11, 2024, 9:00pm

Here is my working running config for the ISP Freedom (simular to KPN)

set interfaces ethernet eth0 hw-id 'xx:xx:xx:xx:xx:36'
set interfaces ethernet eth0 vif 10 address 'xxx.xxx.178.1/29'
set interfaces ethernet eth0 vif 80 address 'xxx.xxx.178.9/29'
set interfaces ethernet eth1 description 'WAN'
set interfaces ethernet eth1 hw-id 'xx:xx:xx:xx:xx:ae'
set interfaces ethernet eth1 mtu '1512'
set interfaces ethernet eth1 vif 4 address 'dhcp'
set interfaces ethernet eth1 vif 4 dhcp-options default-route-distance '254'
set interfaces ethernet eth1 vif 4 dhcp-options no-default-route
set interfaces ethernet eth1 vif 4 dhcp-options user-class '121'
set interfaces ethernet eth1 vif 4 dhcp-options vendor-class-id 'IPTV_RG'
set interfaces ethernet eth1 vif 4 ip source-validation 'loose'
set interfaces ethernet eth1 vif 6 mtu '1500'
set interfaces loopback lo
set interfaces pppoe pppoe1 authentication password xxxxxx
set interfaces pppoe pppoe1 authentication username xxxxxx
set interfaces pppoe pppoe1 ip adjust-mss 'clamp-mss-to-pmtu'
set interfaces pppoe pppoe1 mtu '1492'
set interfaces pppoe pppoe1 source-interface 'eth1.6'
set nat source rule 10 log
set nat source rule 10 outbound-interface name 'pppoe1'
set nat source rule 10 translation address 'masquerade'
set nat source rule 80 outbound-interface name 'eth1.4'
set nat source rule 80 source address 'xxx.xxx.178.8/29'
set nat source rule 80 translation address 'masquerade'
set protocols igmp-proxy interface eth0.80 role 'downstream'
set protocols igmp-proxy interface eth0.80 threshold '1'
set protocols igmp-proxy interface eth1.4 alt-subnet 'xxx.xxx.0.0/0'
set protocols igmp-proxy interface eth1.4 role 'upstream'
set protocols igmp-proxy interface eth1.4 threshold '1'
set protocols static route xxx.xxx.176.0/20 next-hop xxx.xxx.178.2
set service dhcp-server hostfile-update
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.178.8/29 lease '86400'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.178.8/29 option default-router 'xxx.xxx.178.9'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.178.8/29 option domain-name xxxxxx
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.178.8/29 option name-server 'xxx.xxx.178.1'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.178.8/29 range 0 start 'xxx.xxx.178.10'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.178.8/29 range 0 stop 'xxx.xxx.178.14'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.178.8/29 subnet-id '80'
set service dns forwarding allow-from 'xxx.xxx.0.0/0'
set service dns forwarding listen-address 'xxx.xxx.178.1'
set service dns forwarding listen-address 'xxx.xxx.0.1'
set service dns forwarding listen-address '::1'
set service dns forwarding system
set service ntp allow-client xxxxxx 'xxx.xxx.0.0/8'
set service ntp allow-client xxxxxx 'xxx.xxx.0.0/16'
set service ntp allow-client xxxxxx 'xxx.xxx.0.0/8'
set service ntp allow-client xxxxxx 'xxx.xxx.0.0/12'
set service ntp allow-client xxxxxx 'xxx.xxx.0.0/16'
set service ntp allow-client xxxxxx '::1/128'
set service ntp allow-client xxxxxx 'fe80::/10'
set service ntp allow-client xxxxxx 'fc00::/7'
set service ntp listen-address 'xxx.xxx.178.1'
set service ntp server xxxxx.tld
set service ntp server xxxxx.tld
set service ntp server xxxxx.tld
set service ssh listen-address 'xxx.xxx.178.1'
set system config-management commit-revisions '100'
set system conntrack modules ftp
set system conntrack modules h323
set system conntrack modules nfs
set system conntrack modules pptp
set system conntrack modules rtsp
set system conntrack modules sip
set system conntrack modules sqlnet
set system conntrack modules tftp
set system console device ttyS0 speed '115200'
set system domain-name xxxxxx
set system host-name xxxxxx
set system login user xxxxxx authentication encrypted-password xxxxxx
set system login user xxxxxx authentication plaintext-password xxxxxx
set system login user xxxxxx authentication public-keys [email protected] key xxxxxx
set system login user xxxxxx authentication public-keys [email protected] type 'ecdsa-sha2-nistp256'
set system syslog global facility all level 'info'
set system syslog global facility local7 level 'debug'
set system time-zone 'Europe/Amsterdam'

trombone · November 11, 2024, 9:25pm

what do you mean with this? As I understand, mss-clamping only works for TCP, so I’m still stuck with sub-optimal MTU settings for non-TCP (esp. UDP) if I rely on that, correct?

trombone · November 11, 2024, 9:26pm

Thanks! Will certainly use that as template to compare against

Apachez · November 11, 2024, 11:13pm

Because the MSS thingy only applies for TCP-traffic and not UDP-traffic.

With UDP the application is supposed to figure things out on itself which most UDP applications simply doesnt.

Samt with PMTU (Path MTU discovery) where TCP honor what PMTU figured out while UDP doesnt.

Typical usecase is if you try to PXE boot using TFTP server over an encrypted network where the regular max packetsize is 1500 bytes but due to encryption your are down to 1438 or whatever. If the TFTP-server sending the packets is configured with MTU 1500 it will send 1500 bytes packets which if you are unlucky they will become fragmented (unless DF flag is set) which means that the TFTP client will not be able to reconstruct the packets since it lacks ability to defragment incoming traffic. Workaround here is to call it a day and set the server MTU to 1400 bytes (or technically 1438 if you want to microoptimize and maximize theoretical throughput performance).

Also looking at your config you have set WAN to L3-MTU of 1512 bytes which is a bit… odd… Are you sure that your ISP really do 1512 bytes as L3-MTU?

marc_s · November 12, 2024, 6:12am

I can confirm that KPN fiber in the Netherlands increase WAN MTU to 1512 to specifically allow for PPPoE to be set to 1500. The PPPoE session runs over a VLAN, that tag uses 4 bytes, the PPPoE 8, so 1500+4+8=1512.

Odd but true

trombone · November 12, 2024, 10:34pm

I did some more testing, and the minimal working set I have is:

# Working
set interfaces pppoe pppoe0 ip adjust-mss 'clamp-mss-to-pmtu'
set interfaces pppoe pppoe0 mtu '1492'

I have not found a working setup with PPPoE MTU at 1500:

# Not working, fast.com does not load
set interfaces pppoe pppoe0 mtu '1500'

# Not working, fast.com does not load
set interfaces pppoe pppoe0 ip adjust-mss 'clamp-mss-to-pmtu'
set interfaces pppoe pppoe0 mtu '1500'

# Not working, fast.com does not load
set interfaces ethernet eth1 mtu '1512'
set interfaces ethernet eth1 vif 6 mtu '1508'
set interfaces pppoe pppoe0 ip adjust-mss 'clamp-mss-to-pmtu'
set interfaces pppoe pppoe0 mtu '1500'

@diedvdyk based on above, you could drop your eth1 MTU, or at least lower your eth1 MTU to 1504 (instead of 1512)
@marc_s: I recognize what you’re saying, but somehow I can’t get 1500 to work.

I’m not sure how VyOS internally translates MTU (perhaps there’s already some accounting for PPPoE or something?). I’m also still puzzled that I need mss-clamping, while on Mikrotik I didnt need that (perhaps because there my MTU was 1500?). Any ideas are welcome, the performance is not the key problem but ideally I’d not use the mss clamping for higher (perceived?) compatibility.

tjh · November 13, 2024, 1:59am

The ISP I work for, we had to enable the RFC for PPPoE MTU greater than 1492.

Before we moved to different BNGs (and I now get to use DHCP), I had perfectly working PPPoE of 1500 using the “mtu 1500” command, with a 1508 MTU on the underlying Ethernet interface.

But, I could also remove all that and have a perfectly working Internet connection with no requirement to mess with MSS clamping. The only place I clamp MSS at the moment is on Wireguard Interface and I probably don’t even need to do that, but I do do little data over them and it works, so I haven’t fiddled with it.

What happens if you just remove any MTU setting? What does then
“show interface pppoe pppoe” tell you your MTU is? It should be 1492.

Is the MRU matching as well? You could try manually setting MTU and MRU both to 1492.

Finally, what’s the MTU of your bridge? That isn’t the problem bit is it?

trombone · November 13, 2024, 9:15am

Thanks tjh, good leads, let me look at it when the household is asleep

How do I enable rfc4638? My ISP is telling me the MTU is 1500 (‘Maximale pakket grote (mtu) 1500 bytes (rfc4638)’)
What do you mean with ‘remove all that’, i.e. remove all mtu/mss settings? I can try that as well and check what mtu reported.
I can’t set MRU on my VyOS version
VyOS is also my bridge, I haven’t set any MTU so I assume it’s the default 1500.

marc_s · November 13, 2024, 1:56pm

Another thought. You say that you use a virtualized VyOS installation. Have you accounted for the correct MTUs on the hypervisor side of things?

tjh · November 13, 2024, 5:21pm

Oh yes this is a very, very good point! My VyOS is virtualised too, I forgot to mention I make sure to setup very large MTUs on all the physical ports/bridges when I had my 1500 PPPoE working.

Also: Try a later build of VyOS!