This old bug i was referencing is talking about VyOS on AWS, which runs on Xen also from what I recall, so not suprized this is coming up on my own Xen.
[06:51 xen1 ~]# netstat -i
Kernel Interface table
Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 9000 1006030681 0 0 0 74970818 0 0 0 BMRU
eth1 9000 0 0 0 0 0 0 0 0 BMU
eth2 9000 609145160 0 0 0 735455697 0 0 0 BMRU
eth3 9000 0 0 0 0 0 0 0 0 BMU
eth2.2 9000 0 0 0 0 2240382 0 0 0 BMRU
eth2.3 9000 0 0 0 0 1180813 0 0 0 BMRU
eth2.120 9000 50670144 0 0 0 54883148 0 0 0 BMRU
eth2.121 9000 553713 0 0 0 4262 0 0 0 BMRU
eth2.161 9000 0 0 0 0 572587 0 0 0 BMRU
eth2.162 9000 0 0 0 0 570766 0 0 0 BMRU
eth2.163 9000 0 0 0 0 536892 0 0 0 BMRU
eth2.500 9000 15715220 0 0 0 11923650 0 0 0 BMRU
eth2.161 9000 0 0 0 0 240987 0 0 0 BMRU
lo 65536 5634463 0 0 0 5634463 0 0 0 LRU
vif10.0 9000 31636610 0 0 0 48742707 0 0 0 BMORU
vif11.3 9000 17429960 0 0 0 32153482 0 0 0 BMORU
vif12.3 9000 1148213 0 0 0 19038720 0 0 0 BMORU
vif13.0 9000 611289 0 0 0 1537666 0 0 0 BMORU
vif15.0 9000 507773 0 0 0 18714269 0 0 0 BMORU
vif15.1 9000 9 0 0 0 608667 0 0 0 BMORU
vif15.2 9000 4262 0 0 0 553688 0 0 0 BMORU
vif25.0 9000 353316087 0 0 0 417722773 0 0 0 BMORU
vif25.1 9000 536181 0 0 0 101196 0 0 0 BMORU
vif25.2 9000 290614 0 0 0 507603 0 0 0 BMORU
vif25.3 9000 396930378 0 0 0 58636142 0 0 0 BMORU
vif25.4 9000 25700 0 0 0 216940 0 0 0 BMORU
vif27.0 9000 523928 0 0 0 1848803 0 0 0 BMORU
vif3.0 9000 225652326 0 0 0 237752225 0 0 0 BMORU
vif30.0 9000 2483145 0 0 0 3609470 0 0 0 BMORU
vif31.0 9000 30609262 0 0 0 220969966 0 0 0 BMORU
vif33.0 9000 27941473 0 0 0 176375412 0 0 0 BMORU
vif4.0 9000 1673304 0 0 0 1316065 0 0 0 BMORU
vif47.0 9000 2703754 0 0 0 15241547 0 0 0 BMORU
vif48.0 9000 1548621 0 0 0 14686822 0 0 0 BMORU
vif49.0 9000 165351181 0 0 0 177161995 0 0 0 BMORU
vif53.0 9000 3601101 0 0 0 15557479 0 0 0 BMORU
vif53.1 9000 52465 0 0 0 413667 0 0 0 BMORU
vif53.2 9000 1179489 0 0 0 543974 0 0 0 BMORU
vif53.3 9000 58257 0 0 0 419358 0 0 0 BMORU
vif53.4 9000 262846 0 0 0 86307 0 0 0 BMORU
vif54.0 9000 8088 0 0 0 886336 0 0 0 BMORU
vif58.0 9000 1474 0 0 0 13996680 0 0 0 BMORU
vif59.0 1500 54883140 0 0 0 62037358 0 0 0 BMORU
vif70.0 1500 11923650 0 0 0 15715220 0 0 0 BMORU
vif75.0 9000 16018 0 0 0 937911 0 0 0 BMORU
vif75.1 9000 146130608 0 0 0 49231514 0 0 0 BMORU
vif75.2 9000 41171697 0 0 0 15267499 0 0 0 BMORU
vif75.3 9000 68 0 0 0 934622 0 0 0 BMORU
vif75.4 9000 11229 0 0 0 374432 0 0 0 BMORU
vif76.0 9000 28159 0 0 0 941674 0 0 0 BMORU
vif76.1 9000 106834 0 0 0 7500072 0 0 0 BMORU
vif76.2 9000 29 0 0 0 7430551 0 0 0 BMORU
vif76.3 9000 293945 0 0 0 819681 0 0 0 BMORU
vif76.4 9000 606 0 0 0 371497 0 0 0 BMORU
vif78.0 9000 563148 0 0 0 1078773 0 0 0 BMORU
vif78.1 9000 723157 0 0 0 8481730 0 0 0 BMORU
vif78.2 9000 18 0 0 0 7428637 0 0 0 BMORU
vif78.3 9000 18 0 0 0 933978 0 0 0 BMORU
vif78.4 9000 156 0 0 0 371331 0 0 0 BMORU
vif79.0 9000 60607 0 0 0 286727 0 0 0 BMORU
vif8.0 9000 48113 0 0 0 612195 0 0 0 BMORU
vif81.0 9000 68020 0 0 0 788626 0 0 0 BMORU
vif81.1 9000 3286000 0 0 0 8100669 0 0 0 BMORU
vif82.0 9000 28797 0 0 0 758798 0 0 0 BMORU
vif82.1 9000 588157 0 0 0 5447598 0 0 0 BMORU
vif83.0 9000 271153 0 0 0 930145 0 0 0 BMORU
vif83.1 9000 1092463 0 0 0 5701406 0 0 0 BMORU
vif84.0 9000 33498 0 0 0 762033 0 0 0 BMORU
vif84.1 9000 612289 0 0 0 5469333 0 0 0 BMORU
vif94.0 9000 3714 0 0 0 23301 0 0 0 BMORU
vif94.1 9000 18341 0 0 0 150345 0 0 0 BMORU
vif94.2 9000 4143 0 0 0 128139 0 0 0 BMORU
vif94.3 9000 14 0 0 0 20960 0 0 0 BMORU
vif94.4 9000 9556 0 0 0 5083 0 0 0 BMORU
xapi0 9000 2240312 0 0 0 0 0 0 0 BMRU
xapi1 9000 1180771 0 0 0 0 0 0 0 BMRU
xapi2 9000 570754 0 0 0 0 0 0 0 BMRU
xapi3 1500 608710 0 0 0 0 0 0 0 BMRU
xapi4 9000 551920 0 0 0 0 0 0 0 BMRU
xapi5 1500 2 0 0 0 0 0 0 0 BMRU
xapi7 9000 536882 0 0 0 0 0 0 0 BMRU
xapi8 9000 240987 0 0 0 0 0 0 0 BMRU
xapi20 9000 572575 0 0 0 0 0 0 0 BMRU
xenbr0 9000 74890691 0 0 0 64010098 0 0 0 BMRU
xenbr1 9000 0 0 0 0 0 0 0 0 BMU
xenbr2 9000 52068178 0 0 0 46891629 0 0 0 BMRU
xenbr3 9000 0 0 0 0 0 0 0 0 BMU
Theres no packet drops on any interfaces in the hypervisor.
Also, the show hardware pci command above was run while I had the VM in HVM/PV mode for testing. I have reverted it back to full PV now (like I want it) and now when I run show hardware pci I get zero output.
also, i changed the number of vCPU’s to 5 now (1 per NIC), same results.
I don’t have anything set interfaces ethernet ethx smp-affinity, but i tried that before and it still happens.
also commands like set interfaces ethernet ethx mtu 9000 don’t work, but I would like it to support jumbo frames.
I was running full PV with SR-IOV, and everything was working great until I ran into a complication…
When using SR-IOV, the VMs will not live migrate. I do not need to live migrate my vyos boxes but I need to be able to live migrate my other VMs/servers… and I found out the hard way that when you have vyos on SR-IOV and you have VMs on virtual interfaces that are not SR-IOV, the only thing that gets thru is DHCP, after that, the VMs don’t get any ARP traffic from vyos. So I had to scrap SR-IOV for now, until such time that I can have these hypervisors running all vyos instance and all the VM servers are running on other hypervisors. That will happen eventually as I scale out, but for now, the 4 physical boxes that I have need to have SR-IOV off.