After running a while the default routing failed on vyos 1.4 epa1

Hi team,
After upgrade vyos 1.3.6 to vyos 1.4.0 epa 1 . The network of all computers in LAN is fine. But on vyos router , the default routing iis broken.

gw-192-168-11-1:~$ ping 1.1.1.1
/bin/ping: connect: Network is unreachable

Do you use DHCP/PPP as the default or static route?

yes!but in 1.3.x is fine, no such issue.

Needs examples of configuration and logs.
So it is unclear which configuration you have and what steps you should take to reproduce it.
Add a bug report on https://vyos.dev if you can reproduce it. The “set of commands” logs/ show routing table before/after bug, and so on are required.

I hear about interface routes in the task T5811 but it is not related to default route.

I will degrade vyos from 1.4 to 1.3.6. 1.4.0 epa1 and epa2 both has such issue.

We can’t help without config/ steps to reproduce

Here is the configuration:

set interfaces bridge br0 address 'xxx.xxx.100.1/24'
set interfaces bridge br0 member interface eth0
set interfaces bridge br0 member interface eth1
set interfaces bridge br0 member interface eth3
set interfaces bridge br0 member interface eth4
set interfaces ethernet eth0 hw-id 'xx:xx:xx:xx:xx:9c'
set interfaces ethernet eth1 duplex 'auto'
set interfaces ethernet eth1 hw-id 'xx:xx:xx:xx:xx:9d'
set interfaces ethernet eth1 offload gro
set interfaces ethernet eth1 offload gso
set interfaces ethernet eth1 offload sg
set interfaces ethernet eth1 offload tso
set interfaces ethernet eth1 speed 'auto'
set interfaces ethernet eth2 address 'dhcp'
set interfaces ethernet eth2 description 'WAN1'
set interfaces ethernet eth2 duplex 'auto'
set interfaces ethernet eth2 hw-id 'xx:xx:xx:xx:xx:9e'
set interfaces ethernet eth2 offload gro
set interfaces ethernet eth2 offload gso
set interfaces ethernet eth2 offload sg
set interfaces ethernet eth2 offload tso
set interfaces ethernet eth2 speed 'auto'
set interfaces ethernet eth3 duplex 'auto'
set interfaces ethernet eth3 hw-id 'xx:xx:xx:xx:xx:9f'
set interfaces ethernet eth3 offload gro
set interfaces ethernet eth3 offload gso
set interfaces ethernet eth3 offload sg
set interfaces ethernet eth3 offload tso
set interfaces ethernet eth3 speed 'auto'
set interfaces ethernet eth4 duplex 'auto'
set interfaces ethernet eth4 hw-id 'xx:xx:xx:xx:xx:a0'
set interfaces ethernet eth4 offload gro
set interfaces ethernet eth4 offload gso
set interfaces ethernet eth4 offload sg
set interfaces ethernet eth4 offload tso
set interfaces ethernet eth4 speed 'auto'
set interfaces ethernet eth5 address 'xxx.xxx.0.100/24'
set interfaces ethernet eth5 duplex 'auto'
set interfaces ethernet eth5 hw-id 'xx:xx:xx:xx:xx:a1'
set interfaces ethernet eth5 offload gro
set interfaces ethernet eth5 offload gso
set interfaces ethernet eth5 offload sg
set interfaces ethernet eth5 offload tso
set interfaces ethernet eth5 speed 'auto'
set interfaces loopback lo address 'xxx.xxx.16.16/32'
set interfaces pppoe pppoe0 authentication password xxxxxx
set interfaces pppoe pppoe0 authentication username xxxxxx
set interfaces pppoe pppoe0 dhcpv6-options pd 0 interface eth0 address '1'
set interfaces pppoe pppoe0 dhcpv6-options pd 0 interface eth0 sla-id '0'
set interfaces pppoe pppoe0 dhcpv6-options pd 0 length '64'
set interfaces pppoe pppoe0 ipv6 address autoconf
set interfaces pppoe pppoe0 mtu '1492'
set interfaces pppoe pppoe0 no-peer-dns
set interfaces pppoe pppoe0 source-interface 'eth2'
set nat source rule 100 outbound-interface name 'pppoe0'
set nat source rule 100 translation address 'masquerade'
set protocols static route xxx.xxx.0.0/0 interface pppoe0
set protocols static route xxx.xxx.1.0/24 next-hop xxx.xxx.100.2
set protocols static route xxx.xxx.151.0/24 interface wg01
set protocols static route xxx.xxx.3.0/24 next-hop xxx.xxx.100.2
set protocols static route xxx.xxx.11.0/24 interface wg01
set service dhcp-server host-decl-name
set service dhcp-server hostfile-update
set service dhcp-server listen-address 'xxx.xxx.100.1'
set service dhcp-server shared-network-name xxxxxx description 'GW-LAN'
set service dhcp-server shared-network-name xxxxxx domain-name xxxxxx
set service dhcp-server shared-network-name xxxxxx domain-search xxxxxx
set service dhcp-server shared-network-name xxxxxx name-server 'xxx.xxx.19.40'
set service dhcp-server shared-network-name xxxxxx name-server 'xxx.xxx.19.50'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.100.0/24 default-router 'xxx.xxx.100.1'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.100.0/24 range 0 start 'xxx.xxx.100.50'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.100.0/24 range 0 stop 'xxx.xxx.100.254'
set service dhcp-server shared-network-name xxxxxx description 'lan'
set service dhcp-server shared-network-name xxxxxx domain-name xxxxxx
set service dhcp-server shared-network-name xxxxxx domain-search xxxxxx
set service dhcp-server shared-network-name xxxxxx name-server 'xxx.xxx.100.1'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.3.0/24 default-router 'xxx.xxx.3.1'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.3.0/24 name-server 'xxx.xxx.19.40'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.3.0/24 name-server 'xxx.xxx.19.50'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.3.0/24 range 0 start 'xxx.xxx.3.10'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.3.0/24 range 0 stop 'xxx.xxx.3.254'
set service dhcp-server shared-network-name xxxxxx description 'wifi'
set service dhcp-server shared-network-name xxxxxx domain-name xxxxxx
set service dhcp-server shared-network-name xxxxxx domain-search xxxxxx
set service dhcp-server shared-network-name xxxxxx name-server 'xxx.xxx.100.1'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.1.0/24 default-router 'xxx.xxx.1.1'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.1.0/24 name-server 'xxx.xxx.19.40'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.1.0/24 name-server 'xxx.xxx.19.50'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.1.0/24 range 0 start 'xxx.xxx.1.10'
set service dhcp-server shared-network-name xxxxxx subnet xxx.xxx.1.0/24 range 0 stop 'xxx.xxx.1.254'
set service dns dynamic name service-cloudflare-pppoe0 address interface 'pppoe0'
set service dns dynamic name service-cloudflare-pppoe0 host-name xxxxxx
set service dns dynamic name service-cloudflare-pppoe0 password xxxxxx
set service dns dynamic name service-cloudflare-pppoe0 protocol 'cloudflare'
set service dns dynamic name service-cloudflare-pppoe0 username xxxxxx
set service dns dynamic name service-cloudflare-pppoe0 zone 'xxx.xxx'
set service dns dynamic name service-dyndns-pppoe0 address interface 'pppoe0'
set service dns dynamic name service-dyndns-pppoe0 host-name xxxxxx
set service dns dynamic name service-dyndns-pppoe0 password xxxxxx
set service dns dynamic name service-dyndns-pppoe0 protocol 'dyndns2'
set service dns dynamic name service-dyndns-pppoe0 server 'dyn.dns.he.net'
set service dns dynamic name service-dyndns-pppoe0 username xxxxxx
set service dns forwarding allow-from 'xxx.xxx.0.0/0'
set service dns forwarding cache-size '10000'
set service dns forwarding listen-address 'xxx.xxx.100.1'
set service dns forwarding name-server xxx.xxx.1.4
set service dns forwarding name-server xxx.xxx.114.114
set service dns forwarding name-server xxx.xxx.29.29
set service dns forwarding name-server xxx.xxx.19.50
set service https
set service lldp interface all
set service lldp interface eth0
set service ntp allow-client xxxxxx 'xxx.xxx.0.0/0'
set service ntp allow-client xxxxxx '::/0'
set service ntp server xxxxx.tld
set service ntp server xxxxx.tld
set service ntp server xxxxx.tld
set service router-advert interface eth0 default-lifetime '300'
set service router-advert interface eth0 default-preference 'high'
set service router-advert interface eth0 hop-limit '64'
set service router-advert interface eth0 interval max '30'
set service router-advert interface eth0 link-mtu '1452'
set service router-advert interface eth0 other-config-flag
set service router-advert interface eth0 prefix ::/64 preferred-lifetime '300'
set service router-advert interface eth0 prefix ::/64 valid-lifetime '172800'
set service router-advert interface eth0 reachable-time '900000'
set service router-advert interface eth0 retrans-timer '0'
set service snmp community OkayWorld authorization 'ro'
set service ssh disable-host-validation
set service ssh dynamic-protection
set service ssh port '9528'
set system config-management commit-revisions '100'
set system conntrack modules ftp
set system conntrack modules h323
set system conntrack modules nfs
set system conntrack modules pptp
set system conntrack modules sip
set system conntrack modules sqlnet
set system conntrack modules tftp
set system console device ttyS0 speed '9600'
set system host-name xxxxxx
set system login banner post-login '\n\n\tWelcome back!\n\n\n'
set system login banner pre-login '\n\n\tUNAUTHORIZED USE OF THIS SYSTEM IS PROHIBITED!\n\n\n'
set system login user xxxxxx authentication encrypted-password xxxxxx
set system login user xxxxxx authentication plaintext-password xxxxxx
set system login user xxxxxx authentication encrypted-password xxxxxx
set system login user xxxxxx authentication plaintext-password xxxxxx
set system name-server 'xxx.xxx.114.114'
set system syslog global facility all level 'info'
set system syslog global facility local7 level 'debug'
set system task-scheduler task RUN_TASKS_EVERY_MINUTE executable path '/config/scripts/run_tasks_every_minute.sh'
set system task-scheduler task RUN_TASKS_EVERY_MINUTE interval '1m'
set system task-scheduler task daily-reboot crontab-spec '0 5 * * *'
set system task-scheduler task daily-reboot executable path '/sbin/reboot'
set system time-zone 'Asia/Tokyo'

and system version

~$ show version
Version:          VyOS 1.4.0-epa2
Release train:    sagitta

Built by:         Sentrium S.L.
Built on:         Tue 12 Mar 2024 11:58 UTC
Build UUID:       7b60be54-0b8f-4337-aa9e-b6e675942946
Build commit ID:  48f7d41a607707

Architecture:     x86_64
Boot via:         installed image
System type:      bare metal

Hardware vendor:  Default string
Hardware model:   Default string
Hardware S/N:     Default string
Hardware UUID:    03000200-0400-0500-0006-000700080009

Copyright:        VyOS maintainers and contributors

Can you please do some basic debug/troubleshooting steps and information gathering.

  1. Is your PPPoE interface STILL CONNETED when this happens?
  2. Is there anything in the logs that might indicate what the problem is?
  3. If the PPPoE interface is still connceted, what does “show ip route” tell you?

It sounds like your PPPoE Interface has dropped and not reconnecting, but without any detailed logs it’s hard to guess what’s going on.

Is this happening after a while? Or does it just plain not work at all when you upgrade to 1.4?

Please READ all questions and ANSWER all questions when you reply.

Why do you have a dhcp client configured on eth2 if you use a pppoe client on the same interface?
Probably the dhcp client removes the default route after the timeout.
Remove the dhcp client from the eth2 interface and check if you still lose the default route.

2 Likes

Well spotted!! That’s a very bizarre config. Probably your ISP is confused by that and blocks pppoe if it sees a dhcp attempt.

Because It connect to a modern, I want to access portal of modern at the same time. The configuration works at vyos 1.3, but it didn’t works on vyos 1.4 after upgrade it with iso image.

Ok, makes sense.
Does the modem’s DHCP also provide you with a default route?

I know I sound like a broken record but what does your routing table look like.

In vyos 1.3 there was a configure

set interfaces pppoe pppoe0 default-route 'auto'

after upgrade to vyos 1.4. this config losted.

This has been changed to https://docs.vyos.io/en/sagitta/configuration/interfaces/pppoe.html#cfgcmd-set-interfaces-pppoe-interface-no-default-route

We I upgrade to vyos 1.4 epa1/2 it still not working has such issue with pppoe wan. And the network is slow. I revert back to vyos 1.3 now.

Have you removed the dhcp client from eth2?

You have configured static default route via pppoe, remove it.

What does slow mean? Probably MTU issue. To fix it configure MSS-Clamping:

set interfaces pppoe pppoe0 ip adjust-mss 'clamp-mss-to-pmtu'

Nope. The configuration in vyos 1.3 is working. directly upgrade to vyos 1.4 by add system image xxx ,it has such issue and also has slow performance issue. So I revert back to vyos 1.3 .
So it is not suitable time to upgrade vyos. VyOS 1.4 need more time to fix bugs.

What bugs do you mean?

I’ve been using 1.4 for almost a year. I have a setup with a pppoe client. And no problems.

https://vyos.dev/T6140

@Viacheslav @tjh
Update the logs in the ticket
https://vyos.dev/T6140#180602