Unable to use BCM57810

I’m trying to upgrade from 1.2.7 to 1.3.0-epa, but the upgrade process breaks due to not being able to start/use a BCM57810 :

upon manually adding the interface I’m greeted with :

Traceback (most recent call last):
File “/usr/libexec/vyos/conf_mode/interfaces-ethernet.py”, line 163, in
apply(c)
File “/usr/libexec/vyos/conf_mode/interfaces-ethernet.py”, line 152, in apply
e.update(ethernet)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/ethernet.py”, line 340, in update
super().update(config)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py”, line 1416, in update
self.set_admin_state(state)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/interface.py”, line 815, in set_admin_state
return self.set_interface(‘admin_state’, state)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/control.py”, line 183, in set_interface
return self._set_command(self.config, name, value)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/control.py”, line 110, in _set_command
return self._command_set[name].get(‘format’, lambda _: _)(self._cmd(cmd))
File “/usr/lib/python3/dist-packages/vyos/ifconfig/control.py”, line 52, in _cmd
return cmd(command, self.debug)
File “/usr/lib/python3/dist-packages/vyos/util.py”, line 161, in cmd
raise OSError(code, feedback)
FileNotFoundError: [Errno 2] failed to run command: ip link set dev eth5 up
returned:
exit code: 2

noteworthy:
cmd ‘ethtool --pause eth5 autoneg on tx on rx on’
returned (out):

returned (err):
rx unmodified, ignoring
tx unmodified, ignoring
Cannot set device pause parameters: Invalid argument
cmd ‘ip link set dev eth5 up’
returned (out):

returned (err):
RTNETLINK answers: Device or resource busy

[[interfaces ethernet eth5]] failed
Commit failed

jan@rout02:~$ sh log kernel | tail -n 3
[ 3655.178801] bnx2x: [bnx2x_state_wait:310(eth5)]timeout waiting for state 1
[ 3655.179185] bnx2x: [bnx2x_setup_queue:8874(eth5)]Queue(0) SETUP failed
[ 3655.179547] bnx2x: [bnx2x_nic_load:2786(eth5)]Setup leading failed!

So does that look like vyos is trying to set something it shouldn’t ?

thanks,
Jan

Which exactly EPA version?

show version

jan@rout02:~$ sh version

Version: VyOS 1.3.0-epa3
Release train: equuleus

Built by: Sentrium S.L.
Built on: Sun 31 Oct 2021 17:38 UTC
Build UUID: 383e45ad-b32a-4359-8183-9baacc8e69d9
Build commit ID: bb511522cc3bb2-dirty

Architecture: x86_64
Boot via: installed image
System type: VMware guest

Hardware vendor: VMware, Inc.
Hardware model: VMware Virtual Platform
Hardware S/N: VMware-56 4d f7 d2 71 f5 2e ce-df e8 e0 9e ac 2c 87 d3
Hardware UUID: d2f74d56-f571-ce2e-dfe8-e09eac2c87d3

Copyright: VyOS maintainers and contributors

Same behaviour for 1.3.0-epa1, and 1.4-rolling-202111090317 , rebooting back into 1.2.7 gives me a functional device again.

Now, with 1.3.0-epa3 :

root@rout02:~# ethtool -i eth4
driver: bnx2x
version: 1.713.36-0 storm 7.13.11.0
firmware-version: mbi 7.18.27 bc 7.15.77
expansion-rom-version:
bus-info: 0000:0b:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

And the module itself

root@rout02:~# modinfo bnx2x
filename: /lib/modules/5.4.156-amd64-vyos/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko
firmware: bnx2x/bnx2x-e2-7.13.11.0.fw
firmware: bnx2x/bnx2x-e1h-7.13.11.0.fw
firmware: bnx2x/bnx2x-e1-7.13.11.0.fw
version: 1.713.36-0
license: GPL
description: QLogic BCM57710/57711/57711E/57712/57712_MF/57800/57800_MF/57810/57810_MF/57840/57840_MF Driver
author: Eliezer Tamir
srcversion: 775ADF93892D5E4A3564E79
alias: pci:v000014E4d0000163Fsvsdbcsci*
alias: pci:v000014E4d0000163Esvsdbcsci*
alias: pci:v000014E4d0000163Dsvsdbcsci*
alias: pci:v00001077d000016ADsvsdbcsci*
alias: pci:v000014E4d000016ADsvsdbcsci*
alias: pci:v00001077d000016A4svsdbcsci*
alias: pci:v000014E4d000016A4svsdbcsci*
alias: pci:v000014E4d000016ABsvsdbcsci*
alias: pci:v000014E4d000016AFsvsdbcsci*
alias: pci:v000014E4d000016A2svsdbcsci*
alias: pci:v00001077d000016A1svsdbcsci*
alias: pci:v000014E4d000016A1svsdbcsci*
alias: pci:v000014E4d0000168Dsvsdbcsci*
alias: pci:v000014E4d000016AEsvsdbcsci*
alias: pci:v000014E4d0000168Esvsdbcsci*
alias: pci:v000014E4d000016A9svsdbcsci*
alias: pci:v000014E4d000016A5svsdbcsci*
alias: pci:v000014E4d0000168Asvsdbcsci*
alias: pci:v000014E4d0000166Fsvsdbcsci*
alias: pci:v000014E4d00001663svsdbcsci*
alias: pci:v000014E4d00001662svsdbcsci*
alias: pci:v000014E4d00001650svsdbcsci*
alias: pci:v000014E4d0000164Fsvsdbcsci*
alias: pci:v000014E4d0000164Esvsdbcsci*
depends: mdio,libcrc32c
retpoline: Y
intree: Y
name: bnx2x
vermagic: 5.4.156-amd64-vyos SMP mod_unload modversions
parm: num_queues: Set number of queues (default is as a number of CPUs) (int)
parm: disable_tpa: Disable the TPA (LRO) feature (int)
parm: int_mode: Force interrupt mode other than MSI-X (1 INT#x; 2 MSI) (int)
parm: dropless_fc: Pause on exhausted host ring (int)
parm: mrrs: Force Max Read Req Size (0…3) (for debug) (int)
parm: debug: Default debug msglevel (int)

And the physical view

jan@rout02:~$ sh interfaces ethernet eth4 physical
Settings for eth4:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: 10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Port: FIBRE
PHYAD: 1
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: g
Wake-on: g
Current message level: 0x00000000 (0)

    Link detected: no

Ring parameters for eth4:
Pre-set maximums:
RX: 4078
RX Mini: 0
RX Jumbo: 0
TX: 4078
Current hardware settings:
RX: 2039
RX Mini: 0
RX Jumbo: 0
TX: 4078

driver: bnx2x
version: 1.713.36-0 storm 7.13.11.0
firmware-version: mbi 7.18.27 bc 7.15.77
expansion-rom-version:
bus-info: 0000:0b:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

I’ve disabled LLDP on the connected switches, but that doesn’t change the behaviour either.

I see that :

but Debian -- Details of package firmware-bnx2x in buster has bnx2x/bnx2x-e2-7.13.1.0.fw as highest version.

This might be a bit out of scope then, but can I (and how) load the lower version of firmware in any way ?

Ultimately, the issue lies with the combination of the vyos version, the vmware version and said BCM57710 card.

1.2 works with vmware 7.0
1.2 & 1.3 works with vmware 7.0U3

It is unknown whether this is solely tied to the cards firmware, but upgrading vmware fixes the issue for me.

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.