Where is the driver module (sfc.so) for Solarflare NIC card?

billdh2k · February 15, 2021, 7:37am

Newie here. Trying to build a home 10GB switch with one Intel 520-T NIC and one Solarflare SFN5122F. Both are dual ports NICs. Loaded the latest rolling release (Feb 14, 2021). Intel card showed up, but not 5122F. lspci showed the card has been identified - two instances as “Solarflare SFC9020”. But no driver were loaded.

My question - do I have to manually load the driver for Solarflare? If yes, where is the driver module on the disk? The name of the module should be “sfc.so”, but I could not find it. I used $find /lib/modules, but “sfc.so” is not among the output list, from the built-in library “5.10.14-amd64-vyos”.

Thanks,

Bill Huang

Dmitry · February 15, 2021, 8:21am

Hello @billdh2k, could you find device initialization in dmesg output?

sudo dmesg | grep "net\|eth"

Dmitry · February 15, 2021, 8:33am

I can confirm that SFC missing in rolling’s kernel

vyos@vyos:~$ sudo cat /boot/config-5.10.14-amd64-vyos | grep SFC

But it present in stable

vyos@vyos# cat /boot/config-4.19.169-amd64-vyos | grep SFC
CONFIG_SFC=m
CONFIG_SFC_MCDI_MON=y
CONFIG_SFC_SRIOV=y
CONFIG_SFC_MCDI_LOGGING=y
CONFIG_SFC_FALCON=m

billdh2k · February 15, 2021, 3:52pm

Thank you for your answer. I will try the stable release. I will be nice if sfc could be added as defaults, as Solarflare NICs (older generations cards) are popular now since there are loads of them on eBay’s with dirt cheap prices.

Thansk!

billdh2k · February 15, 2021, 4:35pm

Could you point to me where to download the iso image of the stable release? Followed a few links without success. Thanks!

Dmitry · February 15, 2021, 6:01pm

You need to build a stable ISO by yourself https://docs.vyos.io/en/latest/contributing/build-vyos.html

Added feature request https://phabricator.vyos.net/T3312

phillipmcmahon · February 15, 2021, 7:11pm

There’s a pull request already in to add the modules to the rolling release. I suspect you’ll see it in the next couple of days.

Dmitry · February 17, 2021, 6:22am

@billdh2k, modules already available on 1.3 beta. 1.4 rolling still on updating stage

vyos@vyos:~$ show version | match Version
Version:          VyOS 1.3-beta-202102160443
vyos@vyos:~$ sudo cat /boot/config-5.4.96-amd64-vyos | grep SFC
CONFIG_SFC=m
CONFIG_SFC_MCDI_MON=y
CONFIG_SFC_SRIOV=y
CONFIG_SFC_MCDI_LOGGING=y
CONFIG_SFC_FALCON=m
vyos@vyos:~$ sudo modinfo sfc
filename:       /lib/modules/5.4.96-amd64-vyos/kernel/drivers/net/ethernet/sfc/sfc.ko
version:        4.1
license:        GPL
description:    Solarflare network driver
author:         Solarflare Communications and Michael Brown <mbrown@fensystems.co.uk>
srcversion:     602DE37E50A8E49A32F9872
alias:          pci:v00001924d00001B03sv*sd*bc*sc*i*
alias:          pci:v00001924d00000B03sv*sd*bc*sc*i*
alias:          pci:v00001924d00001A03sv*sd*bc*sc*i*
alias:          pci:v00001924d00000A03sv*sd*bc*sc*i*
alias:          pci:v00001924d00001923sv*sd*bc*sc*i*
alias:          pci:v00001924d00000923sv*sd*bc*sc*i*
alias:          pci:v00001924d00001903sv*sd*bc*sc*i*
alias:          pci:v00001924d00000903sv*sd*bc*sc*i*
alias:          pci:v00001924d00000813sv*sd*bc*sc*i*
alias:          pci:v00001924d00000803sv*sd*bc*sc*i*
depends:        mdio
retpoline:      Y
intree:         Y
name:           sfc
vermagic:       5.4.96-amd64-vyos SMP mod_unload modversions 
parm:           vf_max_tx_channels:Limit the number of TX channels VFs can use (uint)
parm:           max_vfs:Reduce the number of VFs initialized by the driver (int)
parm:           mcdi_logging_default:Enable MCDI logging on newly-probed functions (bool)
parm:           rx_refill_threshold:RX descriptor ring refill threshold (%) (uint)
parm:           efx_separate_tx_channels:Use separate channels for TX and RX (bool)
parm:           rss_cpus:Number of CPUs to use for Receive-Side Scaling (uint)
parm:           phy_flash_cfg:Set PHYs into reflash mode initially (bool)
parm:           irq_adapt_low_thresh:Threshold score for reducing IRQ moderation (uint)
parm:           irq_adapt_high_thresh:Threshold score for increasing IRQ moderation (uint)
parm:           debug:Bitmapped debugging message enable value (uint)
parm:           interrupt_mode:Interrupt mode (0=>MSIX 1=>MSI 2=>legacy) (uint)

@billdh2k, tell please if it works for you.

billdh2k · February 17, 2021, 6:22am

Installed 1.3 beta. It successfully picked up the Solarflare SFN5322F card. Still playing with the settings and will report back when the “bridge” of four local 10GE ports are working.

Thanks a lot for the quick response!

billdh2k · February 17, 2021, 7:48am

Reporting back that the Solarflare (SFC) driver works great with the latest 1.3 beta. Configured 4x 10Gb ports (mixed 2x Intel 540-T2 ports and 2x Solarflare SNF+ fibers) as a local switch. Performance with Vyos is much faster than pfSense, under identical hardware setup!

Thanks lot for the great software!

billdh2k · February 18, 2021, 10:58pm

Another update.

The log file, from “dmesg”, showed a failed execution command associated with SFC card:
“MC command 0x2a inlen 16 failed rc=-22 (raw=0) arg=0”. Actual msg is attached at the end.

Not sure what it meant. The interface and the bridge appeared to be working just fine. I was also able to set MTU to “9000”, for the bridge and its interface members.

**Tried to move to latest rolling release (20120217): **

The same “MC command …” error persists, each time a config change was committed to the sfc card. Trying change MTU to 9000 for the SFC card (SFN5322F) caused a kernel panic, resulting in “system reboot in 60 seconds” (happens as soon as “commit” was made). After reboot, MTU value stays at default (1500). (I did not capture the error msg, since it was on console screen).

No problem changing MTU for the Intel card (540-T2). If I leave the MTU at default (i.e. no attempt was made to modify for SFC cards), the system appears to work just fine.

######## Attachment. With1.3 beta, showing “MC command failed” error, but still a success with SFC card. Rolling release would cause kernel panic if attempt to modify MTU for sfc card was made.

login as: vyos

Linux vyos 5.4.96-amd64-vyos #1 SMP Mon Feb 15 20:00:31 UTC 2021 x86_64

Last login: Thu Feb 18 21:39:53 2021 from 192.168.2.20
vyos@vyos:~ vyos@vyos:~ sudo dmesg |grep sfc
[ 4.829766] sfc 0000:03:00.0 (unnamed net_device) (uninitialized): Solarflare NIC detected
[ 4.848143] sfc 0000:03:00.0 (unnamed net_device) (uninitialized): Part Number : SFN5322F
[ 4.947647] sfc 0000:03:00.0 (unnamed net_device) (uninitialized): no SR-IOV VFs probed
[ 5.094518] sfc 0000:03:00.1 (unnamed net_device) (uninitialized): Solarflare NIC detected
[ 5.105092] sfc 0000:03:00.1 (unnamed net_device) (uninitialized): Part Number : SFN5322F
[ 5.196755] sfc 0000:03:00.1 (unnamed net_device) (uninitialized): no SR-IOV VFs probed
[ 5.204651] sfc 0000:03:00.1 (unnamed net_device) (uninitialized): no PTP support
[ 23.465837] sfc 0000:03:00.0 rename3: renamed from eth1
[ 23.616857] sfc 0000:03:00.0 eth2: renamed from rename3
[ 34.960469] sfc 0000:03:00.1 eth3: MC command 0x2a inlen 16 failed rc=-22 (raw=0) arg=0
[ 34.972954] sfc 0000:03:00.1 eth3: link up at 10000Mbps full-duplex (MTU 1500)
[ 35.608602] sfc 0000:03:00.0 eth2: MC command 0x2a inlen 16 failed rc=-22 (raw=0) arg=0
[ 35.621779] sfc 0000:03:00.0 eth2: link up at 10000Mbps full-duplex (MTU 1500)
[ 221.981169] sfc 0000:03:00.1 eth3: MC command 0x2a inlen 16 failed rc=-22 (raw=0) arg=0
[ 222.883753] sfc 0000:03:00.0 eth2: MC command 0x2a inlen 16 failed rc=-22 (raw=0) arg=0
vyos@vyos:~$

system · February 20, 2021, 10:52pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.