Intel 82576 NIC resets while using PCI passthrough on ESXi

Hi Team
I want to use vyos on esxi ,but the intel nic resets when i using pci passthrough on esxi .

[   21.053132] igb 0000:0b:00.0 eth1: Reset adapter
[   22.076862] igb 0000:0b:00.1 eth2: Reset adapter
[   23.724403] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   25.149404] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   29.244841] igb 0000:0b:00.0 eth1: Reset adapter
[   31.292822] igb 0000:0b:00.1 eth2: Reset adapter
[   31.858408] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   34.543434] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   36.924903] igb 0000:0b:00.0 eth1: Reset adapter
[   39.525468] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   39.996899] igb 0000:0b:00.1 eth2: Reset adapter
[   43.136467] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   45.116915] igb 0000:0b:00.0 eth1: Reset adapter
[   47.489434] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   49.212899] igb 0000:0b:00.1 eth2: Reset adapter
[   52.331477] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   52.796902] igb 0000:0b:00.0 eth1: Reset adapter
[   55.475477] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   57.916936] igb 0000:0b:00.1 eth2: Reset adapter
[   60.988918] igb 0000:0b:00.0 eth1: Reset adapter
[   61.037449] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   63.266458] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   67.132947] igb 0000:0b:00.1 eth2: Reset adapter
[   69.180902] igb 0000:0b:00.0 eth1: Reset adapter
[   70.309504] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   71.497487] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   75.836890] igb 0000:0b:00.1 eth2: Reset adapter
[   76.860913] igb 0000:0b:00.0 eth1: Reset adapter
[   78.921511] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   79.133500] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   85.052978] igb 0000:0b:00.0 eth1: Reset adapter
[   87.786511] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   89.148945] igb 0000:0b:00.1 eth2: Reset adapter
[   92.277536] igb 0000:0b:00.1 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   93.245009] igb 0000:0b:00.0 eth1: Reset adapter
[   95.605522] igb 0000:0b:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   97.854287] igb 0000:0b:00.1 eth2: Reset adapter
vyos@vyos:~$      
vyos@vyos:~$ show interfaces ethernet eth1 physical 
Settings for eth1:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on (auto)
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: no
Ring parameters for eth1:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             256
RX Mini:        0
RX Jumbo:       0
TX:             256

driver: igb
version: 5.6.0-k
firmware-version: 1.2.1
expansion-rom-version: 
bus-info: 0000:0b:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
vyos@vyos:~$ lspci -vvv
0b:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Physical Slot: 192
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 19
        Region 0: Memory at fd3c0000 (32-bit, non-prefetchable) [size=128K]
        Region 1: Memory at fcc00000 (32-bit, non-prefetchable) [size=4M]
        Region 2: I/O ports at 5000 [size=32]
        Region 3: Memory at fd3fc000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: igb
        Kernel modules: igb

0b:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Physical Slot: 192
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64, Cache Line Size: 64 bytes
        Interrupt: pin B routed to IRQ 16
        Region 0: Memory at fd3a0000 (32-bit, non-prefetchable) [size=128K]
        Region 1: Memory at fc800000 (32-bit, non-prefetchable) [size=4M]
        Region 2: I/O ports at 5020 [size=32]
        Region 3: Memory at fd3f8000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: igb
        Kernel modules: igb

1b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
        Subsystem: VMware VMXNET3 Ethernet Controller
        Physical Slot: 256
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 32 bytes
        Interrupt: pin A routed to IRQ 17
        Region 0: Memory at fc6fc000 (32-bit, non-prefetchable) [size=4K]
        Region 1: Memory at fc6fd000 (32-bit, non-prefetchable) [size=4K]
        Region 2: Memory at fc6fe000 (32-bit, non-prefetchable) [size=8K]
        Region 3: I/O ports at 7000 [size=16]
        [virtual] Expansion ROM at fc600000 [disabled] [size=64K]
        Capabilities: <access denied>
        Kernel driver in use: vmxnet3
        Kernel modules: vmxnet3

vyos@vyos:~$ 

my host is hp gen8 ,and esxi verion is HPE-ESXi-6.5.0-Update3-Gen9plus-650.U3.10.6.0.86 (Hewlett Packard Enterprise)

i have tried vyos-1.4-rolling-202110180513-amd64.iso and vyos-1.3-beta-202110190342-amd64.iso , have the same problem.

what can i do next…

update…

i try Intel 82576 nic on my bm (i5 6500 + b85) pc and use vyos-1.3-beta-202110190342-amd64.iso . it works well . why the nic run vyos on esxi it will be reset ??

i think it is the same problem.
how to solve the problem in vyos ?

i think i soloved this problem when i add pci=nomsi pci=pcie_scan_all in the /boot/grub/grub.cfg

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.