EVPN-VXLAN + ESI-LAG Duplicate Packets

Hi,

I am currently using the latest VyOS nightly with EVPN-VXLAN.

I have VyOS connected to a pair of Juniper QFX5120-48Y switches using standard VLAN interfaces, they are not bonded (ECMP is used).

The basic connectivity looks like this:

This is mostly working except for one problem; any traffic outbound from VyOS to the server appears to be duplicated by VyOS. Using just a simple ping as an example, if I tcpdump on the VyOS side I can see the single ping come in however I can see that two ping replies go out.

After a bit of troubleshooting this only appears to happen if the destination server is using ESI-LAG; hosts that have a single connection or are not using ESI-LAG do not get duplicates sent from VyOS. If I disable one of the switch ports for the ESI-LAG things work as expected. If I disable one of the ports for VyOS the problem still occurs.

The actual VyOS configuration looks like this:

interfaces {
     bridge br0 {
         enable-vlan
         member {
             interface vxlan0 {
             }
         }
         vif 300 {
             address 10.100.65.157/27
         }
     }
     ethernet eth0 {
         mtu 9000
         ring-buffer {
             rx 8192
             tx 8192
         }
         vif 2019 {
             address 10.100.16.36/31
             mtu 9000
         }
     }
     ethernet eth1 {
         mtu 9000
         ring-buffer {
             rx 8192
             tx 8192
         }
         vif 2018 {
             address 10.100.16.38/31
             mtu 9000
         }
     }
     loopback lo {
         address 10.100.0.10/32
     }
     vxlan vxlan0 {
         external
         mtu 1500
         port 4789
         source-address 10.100.0.10
         vlan-to-vni 300 {
             vni 30000
         }
     }
}
protocols {
     bgp {
         address-family {
             ipv4-unicast {
                 maximum-paths {
                     ebgp 16
                     ibgp 16
                 }
                 redistribute {
                     connected {
                     }
                     kernel {
                     }
                     static {
                     }
                 }
             }
             ipv6-unicast {
                 maximum-paths {
                     ebgp 16
                     ibgp 16
                 }
                 redistribute {
                     connected {
                     }
                     kernel {
                     }
                     static {
                     }
                 }
             }
             l2vpn-evpn {
                 advertise-all-vni
                 advertise-default-gw
                 advertise-svi-ip
                 rd 10.100.0.10:1
                 rt-auto-derive
             }
         }
         neighbor 10.100.0.14 {
             address-family {
                 ipv4-flowspec {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             peer-group RouteServers-IPv4
         }
         neighbor 10.100.0.15 {
             address-family {
                 ipv4-flowspec {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             peer-group RouteServers-IPv4
         }
         neighbor 10.100.0.16 {
             address-family {
                 ipv4-flowspec {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             peer-group RouteServers-IPv4
         }
         neighbor 10.100.0.17 {
             address-family {
                 ipv4-flowspec {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             peer-group RouteServers-IPv4
         }
         neighbor 10.100.0.18 {
             address-family {
                 ipv4-flowspec {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             peer-group RouteServers-IPv4
         }
         neighbor 10.100.0.19 {
             address-family {
                 ipv4-flowspec {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             peer-group RouteServers-IPv4
         }
         parameters {
             ebgp-requires-policy
             fast-convergence
             graceful-restart {
             }
             graceful-shutdown
             log-neighbor-changes
             router-id 10.100.0.10
         }
         peer-group RouteServers-IPv4 {
             address-family {
                 ipv4-unicast {
                     addpath-tx-all
                     route-map {
                         export RouteServers-Export-IPv4
                         import RouteServers-Import-IPv4
                     }
                     soft-reconfiguration {
                         inbound
                     }
                 }
                 l2vpn-evpn {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             capability {
             }
             graceful-restart enable
             remote-as 4212300000
             update-source 10.100.0.10
         }
     }
}

On the switch side, the ESI-LAG interface is configured like this:

mtu 9216;
esi {
    00:42:12:30:00:00:00:20:00:03;
    all-active;
}
aggregated-ether-options {
    lacp {
        active;
        system-id 00:00:00:20:00:03;
    }
}
unit 0 {
    family ethernet-switching {
        interface-mode trunk;
        vlan {
            members [ 300-301 ];
        }
    }
}

This FRR issue looks like it may be similar but the kernel is much newer than what was mentioned.

I am wondering if anyone else has come across a similar problem?

ESI support is yet not added to VyOS which means I have yet no experience with it. Bit we could work on it together

1 Like

Sure, I would be happy to help. I joined slack; if you want real time conversation feel free to send me a message there.

What would you like me to do?

2 Likes

Lets move this conversation to Slack

Did you ever solve this? I am having a similar issue, where traffic received by the PE on one member of the LAG is replicated to the other PE.

No unfortunately :frowning:

In the end I ripped out the VXLAN config and just use the switches to translate them to standard VLANs for VyOS.

Hi gbe0, I was wondering if you have tried the latest version? I had a similar issue with duplicate packets, and now they have implemented ESI support, or at least there is a command to show it.