IPoE Server not working anymore after the upgrade of the kernel to 6.1

lferrarotti · January 18, 2023, 1:39pm

Hi all,

based on my latest research, seems that in the latest images after, at least, 1.4-rolling-202301091821 the kernel was upgraded from 5.15.86-amd64-vyos to 6.1.6-amd64-vyos.

Using always the same config as a starting point, just upgrading the images I saw this issue.

When everything is working here some info :

root@VyOS-BSDRP-Router:/home/vyos# ethtool -i eth0
driver: e1000e
version: 5.15.86-amd64-vyos
firmware-version: 1.10-0
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

root@VyOS-BSDRP-Router:/home/vyos# uname -a
Linux VyOS-BSDRP-Router 5.15.86-amd64-vyos #1 SMP Tue Jan 3 07:08:57 UTC 2023 x86_64 GNU/Linux

root@VyOS-BSDRP-Router:/home/vyos# lspci -v | grep Ether
02:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
03:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
04:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
05:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
06:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
09:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
0a:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
0b:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection

Jan 18 14:06:25 VyOS-BSDRP-Router accel-ipoe: cli: tcp: new connection from 127.0.0.1
Jan 18 14:07:30 VyOS-BSDRP-Router accel-ipoe: cli: tcp: new connection from 127.0.0.1
Jan 18 14:07:30 VyOS-BSDRP-Router accel-ipoe: terminate, sig = 15
Jan 18 14:07:30 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.51: ipoe: session finished
Jan 18 14:07:30 VyOS-BSDRP-Router accel-ipoe: ipoe1:eth4.50: ipoe: session finished
Jan 18 14:07:30 VyOS-BSDRP-Router accel-ipoe: ipoe: stop interface eth4.50
Jan 18 14:07:30 VyOS-BSDRP-Router accel-ipoe: ipoe: stop interface eth4.51
Jan 18 14:07:32 VyOS-BSDRP-Router accel-ipoe: ipoe: start interface eth4.50 (shared=1,mode=L2,ifcfg=1,range=172.16.50.0/24,start=dhcpv4,ipv6=1)
Jan 18 14:07:32 VyOS-BSDRP-Router accel-ipoe: ipoe: start interface eth4.51 (shared=1,mode=L2,ifcfg=1,range=172.16.98.0/24,start=dhcpv4,ipv6=1)
Jan 18 14:09:39 VyOS-BSDRP-Router accel-ipoe: eth4.50:: recv [DHCPv4 Discover xid=20779fb8 chaddr=00:0c:29:b7:49:a7 <Message-Type Discover> <Request-IP 172.16.50.2> <Host-Name VyOS-PPPoE-IPoE-Client> <Request-List Subnet,Broadcast,Router,DNS,Classless-Route,Domain-Name,MTU>]
Jan 18 14:09:39 VyOS-BSDRP-Router accel-ipoe: ipoe0:: create interface ipoe0 parent eth4.50
Jan 18 14:09:39 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.50: eth4.50: authentication succeeded
Jan 18 14:09:39 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.50: send [DHCPv4 Offer xid=20779fb8 yiaddr=172.16.50.2 chaddr=00:0c:29:b7:49:a7 <Message-Type Offer> <Server-ID 172.16.50.1> <Lease-Time 600> <T1 300> <T2 525> <Router 172.16.50.1> <Subnet 255.255.255.0> <DNS 10.17.6.120,8.8.8.8>]
Jan 18 14:09:39 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.50: recv [DHCPv4 Request xid=20779fb8 chaddr=00:0c:29:b7:49:a7 <Message-Type Request> <Server-ID 172.16.50.1> <Request-IP 172.16.50.2> <Host-Name VyOS-PPPoE-IPoE-Client> <Request-List Subnet,Broadcast,Router,DNS,Classless-Route,Domain-Name,MTU>]
Jan 18 14:09:39 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.50: ipoe: session started
Jan 18 14:09:39 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.50: send [DHCPv4 Ack xid=20779fb8 yiaddr=172.16.50.2 chaddr=00:0c:29:b7:49:a7 <Message-Type Ack> <Server-ID 172.16.50.1> <Lease-Time 600> <T1 300> <T2 525> <Router 172.16.50.1> <Subnet 255.255.255.0> <DNS 10.17.6.120,8.8.8.8>]
Jan 18 14:09:40 VyOS-BSDRP-Router accel-ipoe: eth4.51:: recv [DHCPv4 Discover xid=14f5a1bd chaddr=00:0c:29:b7:49:a7 <Message-Type Discover> <Request-IP 172.16.98.2> <Host-Name VyOS-PPPoE-IPoE-Client> <Request-List Subnet,Broadcast,Router,DNS,Classless-Route,Domain-Name,MTU>]
Jan 18 14:09:40 VyOS-BSDRP-Router accel-ipoe: ipoe1:: create interface ipoe1 parent eth4.51
Jan 18 14:09:40 VyOS-BSDRP-Router accel-ipoe: ipoe1:eth4.51: eth4.51: authentication succeeded
Jan 18 14:09:40 VyOS-BSDRP-Router accel-ipoe: ipoe1:eth4.51: send [DHCPv4 Offer xid=14f5a1bd yiaddr=172.16.98.2 chaddr=00:0c:29:b7:49:a7 <Message-Type Offer> <Server-ID 172.16.98.1> <Lease-Time 600> <T1 300> <T2 525> <Router 172.16.98.1> <Subnet 255.255.255.0> <DNS 10.17.6.120,8.8.8.8>]
Jan 18 14:09:40 VyOS-BSDRP-Router accel-ipoe: ipoe1:eth4.51: recv [DHCPv4 Request xid=14f5a1bd chaddr=00:0c:29:b7:49:a7 <Message-Type Request> <Server-ID 172.16.98.1> <Request-IP 172.16.98.2> <Host-Name VyOS-PPPoE-IPoE-Client> <Request-List Subnet,Broadcast,Router,DNS,Classless-Route,Domain-Name,MTU>]
Jan 18 14:09:40 VyOS-BSDRP-Router accel-ipoe: ipoe1:eth4.51: ipoe: session started

With VyOS 1.4-rolling-202301171813 here some info :

root@VyOS-BSDRP-Router:/home/vyos# ethtool -i eth0
driver: e1000e
version: 6.1.6-amd64-vyos
firmware-version: 1.10-0
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

root@VyOS-BSDRP-Router:/home/vyos# uname -a
Linux VyOS-BSDRP-Router 6.1.6-amd64-vyos #1 SMP PREEMPT_DYNAMIC Sun Jan 15 08:56:10 UTC 2023 x86_64 GNU/Linux

root@VyOS-BSDRP-Router:/home/vyos# lspci -v | grep Ether
02:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
03:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
04:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
05:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
06:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
09:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
0a:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
0b:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection

Jan 18 14:22:29 VyOS-BSDRP-Router accel-ipoe: cli: tcp: new connection from 127.0.0.1
Jan 18 14:25:03 VyOS-BSDRP-Router accel-ipoe: cli: tcp: new connection from 127.0.0.1
Jan 18 14:25:03 VyOS-BSDRP-Router accel-ipoe: terminate, sig = 15
Jan 18 14:25:03 VyOS-BSDRP-Router accel-ipoe: ipoe: stop interface eth4.50
Jan 18 14:25:03 VyOS-BSDRP-Router accel-ipoe: ipoe: nl_del_iface: Invalid argument
Jan 18 14:25:03 VyOS-BSDRP-Router accel-ipoe: ipoe: stop interface eth4.51
Jan 18 14:25:04 VyOS-BSDRP-Router accel-ipoe: ipoe: start interface eth4.50 (shared=1,mode=L2,ifcfg=1,range=172.16.50.0/24,start=dhcpv4,ipv6=1)
Jan 18 14:25:04 VyOS-BSDRP-Router accel-ipoe: ipoe: start interface eth4.51 (shared=1,mode=L2,ifcfg=1,range=172.16.98.0/24,start=dhcpv4,ipv6=1)
Jan 18 14:25:04 VyOS-BSDRP-Router accel-ipoe: vlan_mon: nl_del_vlan_mon: error talking to kernel
Jan 18 14:27:27 VyOS-BSDRP-Router accel-ipoe: send [DHCPv4 Nak xid=14f5a1bd chaddr=00:0c:29:b7:49:a7 <Message-Type Nak> <Server-ID 255.255.255.255> <Message Session dosn't exist>]
Jan 18 14:27:27 VyOS-BSDRP-Router accel-ipoe: send [DHCPv4 Nak xid=20779fb8 chaddr=00:0c:29:b7:49:a7 <Message-Type Nak> <Server-ID 255.255.255.255> <Message Session dosn't exist>]
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: eth4.51:: recv [DHCPv4 Discover xid=577891e8 chaddr=00:0c:29:b7:49:a7 <Message-Type Discover> <Host-Name VyOS-PPPoE-IPoE-Client> <Request-List Subnet,Broadcast,Router,DNS,Classless-Route,Domain-Name,MTU>]
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: ipoe0:: create interface ipoe0 parent eth4.51
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.51: eth4.51: authentication succeeded
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.51: send [DHCPv4 Offer xid=577891e8 yiaddr=172.16.98.2 chaddr=00:0c:29:b7:49:a7 <Message-Type Offer> <Server-ID 172.16.98.1> <Lease-Time 600> <T1 300> <T2 525> <Router 172.16.98.1> <Subnet 255.255.255.0> <DNS 10.17.6.120,8.8.8.8>]
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.51: recv [DHCPv4 Request xid=577891e8 chaddr=00:0c:29:b7:49:a7 <Message-Type Request> <Server-ID 172.16.98.1> <Request-IP 172.16.98.2> <Host-Name VyOS-PPPoE-IPoE-Client> <Request-List Subnet,Broadcast,Router,DNS,Classless-Route,Domain-Name,MTU>]
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.51: ipoe: nl_modify: Invalid argument
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.51: ipoe: session finished
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: ipoe0:eth4.51: ipoe: nl_delete: Invalid argument
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: eth4.50:: recv [DHCPv4 Discover xid=5abe18e2 chaddr=00:0c:29:b7:49:a7 <Message-Type Discover> <Host-Name VyOS-PPPoE-IPoE-Client> <Request-List Subnet,Broadcast,Router,DNS,Classless-Route,Domain-Name,MTU>]
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: eth4.50:: ipoe: nl_create: Invalid argument
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: eth4.50:: ipoe: failed to create interface
Jan 18 14:27:29 VyOS-BSDRP-Router accel-ipoe: eth4.50:: ipoe: session finished

So probably with the new kernel something in the naming of interfaces or similar was changed.

Also after a manual restart of the service here the important logs and errors :

Jan 18 14:38:35 VyOS-BSDRP-Router accel-ipoe: cli: tcp: new connection from 127.0.0.1
Jan 18 14:38:35 VyOS-BSDRP-Router accel-ipoe: terminate, sig = 15
Jan 18 14:38:35 VyOS-BSDRP-Router accel-ipoe: ipoe: stop interface eth4.50
Jan 18 14:38:35 VyOS-BSDRP-Router accel-ipoe: ipoe: nl_del_iface: Invalid argument
Jan 18 14:38:35 VyOS-BSDRP-Router accel-ipoe: ipoe: stop interface eth4.51
Jan 18 14:38:36 VyOS-BSDRP-Router accel-ipoe: ipoe: nl_del_exclude: Invalid argument
Jan 18 14:38:36 VyOS-BSDRP-Router accel-ipoe: ipoe: nl_del_net: Invalid argument
Jan 18 14:38:36 VyOS-BSDRP-Router accel-ipoe: ipoe: start interface eth4.50 (shared=1,mode=L2,ifcfg=1,range=172.16.50.0/24,start=dhcpv4,ipv6=1)
Jan 18 14:38:36 VyOS-BSDRP-Router accel-ipoe: ipoe: start interface eth4.51 (shared=1,mode=L2,ifcfg=1,range=172.16.98.0/24,start=dhcpv4,ipv6=1)

Best regards,
Luca

Viacheslav · January 18, 2023, 2:10pm

Thank you for the report.
I created the task T4941

c-po · January 23, 2023, 5:17am

Hi Luca,

can you please share your server and client configuration so we can reproduce this in the lab?
We received an upstream patch which I would like to validate.

–
Christian

lferrarotti · January 23, 2023, 8:18am

Hi Christian,

here the config as per request.

Server config :

set interfaces ethernet eth4 description 'VSR-IPoE'
set interfaces ethernet eth4 hw-id 'xx:xx:xx:xx:xx:xx'
set interfaces ethernet eth4 vif 50
set interfaces ethernet eth4 vif 51

set service ipoe-server authentication interface eth4.50 mac 00:0c:29:b7:49:a7
set service ipoe-server authentication interface eth4.50 mac 00:0c:29:f0:be:4c rate-limit download '5000'
set service ipoe-server authentication interface eth4.50 mac 00:0c:29:f0:be:4c rate-limit upload '5000'
set service ipoe-server authentication interface eth4.51 mac 00:0c:29:b7:49:a7 rate-limit download '50000'
set service ipoe-server authentication interface eth4.51 mac 00:0c:29:b7:49:a7 rate-limit upload '50000'
set service ipoe-server authentication mode 'local'
set service ipoe-server client-ipv6-pool delegate xxxx:xxx:xxxx::/48 delegation-prefix '56'
set service ipoe-server client-ipv6-pool xxxx:xxx:xxxx::/48 mask '64'
set service ipoe-server interface eth4.50 client-subnet '172.16.50.0/24'
set service ipoe-server interface eth4.50 mode 'l2'
set service ipoe-server interface eth4.51 client-subnet '172.16.98.0/24'
set service ipoe-server interface eth4.51 mode 'l2'
set service ipoe-server name-server 'xxx.xxx.xxx.xxx'
set service ipoe-server name-server '8.8.8.8'
set service ipoe-server name-server '2001:4860:4860::8888'
set service ipoe-server name-server '2001:4860:4860::8844'

Client config :

set interfaces ethernet eth3 hw-id '00:0c:29:b7:49:a7'
set interfaces ethernet eth3 vif 50 address 'dhcp'
set interfaces ethernet eth3 vif 50 address 'dhcpv6'
set interfaces ethernet eth3 vif 50 description 'VSR-IPoE'
set interfaces ethernet eth3 vif 50 dhcpv6-options pd 0 interface eth2.50 sla-id '1'
set interfaces ethernet eth3 vif 50 vrf 'IPoE'
set interfaces ethernet eth3 vif 51 address 'dhcp'
set interfaces ethernet eth3 vif 51 address 'dhcpv6'
set interfaces ethernet eth3 vif 51 description 'VSR-IPoE-BIS'
set interfaces ethernet eth3 vif 51 dhcpv6-options pd 0 interface eth2.51 sla-id '1'
set interfaces ethernet eth3 vif 51 vrf 'IPoE-BIS'

Actually this config is still working perfectly with : VyOS 1.4-rolling-202301091821

Best Regards,
Luca

Viacheslav · January 24, 2023, 5:34pm

Will be fixed in the next rolling release
Let us know if it works as expected
Thanks

lferrarotti · January 25, 2023, 7:50am

Hi @Viacheslav,
I can confirm that with VyOS 1.4-rolling-202301250317 the config is working as expected and also as before.
So the fix seems to be perfectly working and resolving the issue. At least based on my setup
Thanks and Regards,
Luca

system · January 27, 2023, 7:51am

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.