Has anyone had experience bonding with a Cisco switch? I’m using a pair of stacked 3750G-48 switches and am having some difficulty. The bonds come up but don’t pass certain traffic. From inside the network I can get out to any one of a number of websites, etc. From outside the network I can SSH into various devices as well as connect to internal websites. I also have several IPsec tunnels on Vyos that are up. However, IP phones cannot register to a softswitch that is inside. Or if they register, I RTP packets won’t traverse the network.
If I connect just a single interface to the switch with vif’s on that interface, everything works perfectly.
ethernet eth3 {
bond-group bond0
description "Left Ethernet - switch port g1/0/1"
duplex auto
hw-id a0:36:9f:02:63:32
mtu 9000
smp_affinity auto
speed auto
}
ethernet eth4 {
bond-group bond0
description "Center Left Ethernet - switch port g2/0/1"
duplex auto
hw-id a0:36:9f:02:63:33
mtu 9000
smp_affinity auto
speed auto
}
I have almost an identical setup, with pretty much the exact configuration (except I don’t have mtu 9000 set). Unfortunately, I can’t help troubleshoot very much as this setup is still using Vyatta VC6.2-2011.02.09, so it’s not quite the same. I actually don’t think it’s a problem with the LACP links. I would expect there to be widespread failure if they weren’t working properly, this feels like a layer 3 problem but I have no proof. I would try shut on one of the the cisco ports, and see if the behavior persists. If it goes away I think we can say that it’s something wrong with the agg ports.
I have the same result whether using Vyatta 6.5 or Vyos 1.1.5. As to it being a layer 3 issue, from inside the network I can visit websites, get my mail, ssh to various devices and that always works. From outside the network, everything seems to work except RTP and (sometimes) SIP registrations.
I’ll try disconnecting one of the ports and see what that does.
What is the softswitch? If you are sure that layer3 is working (firewall,routing,nat, etc.) then I might look at layer2. Pull out wireshark/tshark/tcpdump and try to see the layer 2 traffic flowing in and out of the switch, the vyos router, and the softswitch. I think LACP uses virtual mac addresses to present channel-groups to switch and host. Maybe there is something there that the softswitch isn’t sending/receiving the frames because of mac address/ARP strangeness. What’s the topology? are both the softswitch and the vyos router plugged into the cisco switchstack? You could try taking VRRP out of the equation to eliminate that. You may need a transparent bridge and some frame/packet traces to figure this one out.
No firewalls at all. Traffic takes the same path if I eliminate LACP and just use VIFs and VRRP on an ethernet interface. In that instance, everything works fine. Only when I go to a bond on Vyos and a port channel on the Cisco do I not pass RTP.
The softswitch lives on a Xenserver and as stated above, all of this works fine when using a straight ethernet interface with VIFs. Only when I go to a bonded interface with a port channel on the switch do I run into trouble.
Everything (Vyos, softswitch, etc.) is connected to the 3750s.
As an aside, I have 2 SANs running Ubuntu 14.04.2 and they connect to the switches with LACP with no problems, although they aren’t running RTP
