Configuring L2VPN EVPN VXLAN in FRR

I see FRR 7.2 in release 1.2.4, but I can not find any information about configuring EVPN address family in BGP and using that for L2 NLRI between VTEPs. Is this supported in VyOS, and is there any documentation?

Hi JDL!

EVPN is not supported yet in the vyos, for now only manually configured P2P VXLAN tunnels are supported.
it’s on the todo list, but i don’t think it will be here until after the FRR parser is rewritten.

Is it possible to manually configure FRR to get this working?

In theory it is possible to configure it manually using vtysh commands, BUT you need to make sure your config is imported again on reload because the configuration is not saved on reload(post-boot config script or something might help). and it might be overwritten when commiting new config as the configuration engine might reapply config.

@runar @JDL Even if I configure evpn using vtysh how do I disable mac learning on vxlan iface so that it does not broadcast arp?

EVPN is not available in 1.4 rolling release

Should this read “now available”?

You can configure it

vyos@r-roll01# set protocols bgp 65001 address-family l2vpn-evpn 
Possible completions:
   advertise-all-vni
                Advertise All local VNIs
   advertise-default-gw
                Advertise All default g/w mac-ip routes in EVPN
   advertise-pip
                EVPN system primary IP
   advertise-svi-ip
                Advertise svi mac-ip routes in EVPN
 > flooding     Specify handling for BUM packets
   rd           Route Distinguisher
 > route-target Route Target
   rt-auto-derive
                Auto derivation of Route Target (RFC8365)
+> vni          VXLAN Network Identifier

I built a VXLAN network with a centralized L3 gateway. my VYOS VTEP filter arp reply packets from L3 gateway placed on other remote VTEP.

*> [2]:[0]:[48]:[34:0a:98:2f:ef:f1]
172.20.24.6
RT:1:13 ET:8 MM:0, sticky MAC
*> [2]:[0]:[48]:[34:0a:98:2f:ef:f1]:[32]:[172.20.13.252]
172.20.24.6
RT:1:13 ET:8

vbash-4.1# tcpdump -i eth0 -nn host 172.20.24.6 and not icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:11:29.430271 IP 172.20.24.120.38700 > 172.20.24.6.4789: VXLAN, flags [I] (0x08), vni 10013
ARP, Request who-has 172.20.13.252 tell 172.20.13.83, length 28
10:11:29.431164 IP 172.20.24.6.38700 > 172.20.24.120.4789: VXLAN, flags [I] (0x08), vni 10013
ARP, Reply 172.20.13.252 is-at 34:0a:98:2f:ef:f1, length 52
10:11:30.454270 IP 172.20.24.120.38700 > 172.20.24.6.4789: VXLAN, flags [I] (0x08), vni 10013
ARP, Request who-has 172.20.13.252 tell 172.20.13.83, length 28
10:11:30.455158 IP 172.20.24.6.38700 > 172.20.24.120.4789: VXLAN, flags [I] (0x08), vni 10013
ARP, Reply 172.20.13.252 is-at 34:0a:98:2f:ef:f1, length 52
10:11:31.479411 IP 172.20.24.120.38700 > 172.20.24.6.4789: VXLAN, flags [I] (0x08), vni 10013

vbash-4.1# tcpdump -i vxlan10013 -nn arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vxlan10013, link-type EN10MB (Ethernet), capture size 262144 bytes
10:11:17.142174 ARP, Request who-has 172.20.13.252 tell 172.20.13.83, length 28
10:11:18.166138 ARP, Request who-has 172.20.13.252 tell 172.20.13.83, length 28
10:11:19.191157 ARP, Request who-has 172.20.13.252 tell 172.20.13.83, length 28
10:11:20.214201 ARP, Request who-has 172.20.13.252 tell 172.20.13.83, length 28

others arp are comming normal. the problem only with arp from “sticky MAC”

Can you share your vxlan configuration?
And which version do you use?

vyos@vyos# run show system ima
The system currently has the following image(s) installed:

1: 1.4-rolling-202103251004 (default boot) (running image)
2: 1.4-rolling-202103230217

vyos@vyos# run show configuration commands | strip-private
set interfaces bridge br12 ip
set interfaces bridge br12 member interface eth1.12
set interfaces bridge br12 member interface vxlan10012
set interfaces bridge br12 mtu ‘9000’
set interfaces bridge br13 ip
set interfaces bridge br13 member interface eth1.13
set interfaces bridge br13 member interface vxlan10013
set interfaces bridge br13 mtu ‘9000’
set interfaces dummy dum0 address ‘xxx.xxx.24.120/32’
set interfaces ethernet eth0 address ‘xxx.xxx.24.211/31’
set interfaces ethernet eth0 description ‘Uplink’
set interfaces ethernet eth0 ip disable-arp-filter
set interfaces ethernet eth0 mtu ‘9100’
set interfaces ethernet eth1 description ‘VXLAN-NET’
set interfaces ethernet eth1 mtu ‘9100’
set interfaces ethernet eth1 vif 12 mtu ‘2000’
set interfaces ethernet eth1 vif 13 mtu ‘2000’
set interfaces loopback lo
set interfaces vxlan vxlan10012 ip
set interfaces vxlan vxlan10012 mtu ‘1550’
set interfaces vxlan vxlan10012 parameters nolearning
set interfaces vxlan vxlan10012 port ‘4789’
set interfaces vxlan vxlan10012 source-address ‘xxx.xxx.24.120’
set interfaces vxlan vxlan10012 vni ‘10012’
set interfaces vxlan vxlan10013 ip
set interfaces vxlan vxlan10013 mtu ‘1550’
set interfaces vxlan vxlan10013 parameters nolearning
set interfaces vxlan vxlan10013 port ‘4789’
set interfaces vxlan vxlan10013 source-address ‘xxx.xxx.24.120’
set interfaces vxlan vxlan10013 vni ‘10013’
set policy route-map permit rule 1 action ‘permit’
set protocols bgp 65151 address-family l2vpn-evpn advertise-all-vni
set protocols bgp 65151 address-family l2vpn-evpn vni 10012 rd ‘xxx.xxx.24.120:12’
set protocols bgp 65151 address-family l2vpn-evpn vni 10012 route-target both ‘1:12’
set protocols bgp 65151 address-family l2vpn-evpn vni 10012 route-target export ‘1:12’
set protocols bgp 65151 address-family l2vpn-evpn vni 10012 route-target import ‘1:12’
set protocols bgp 65151 address-family l2vpn-evpn vni 10013 rd ‘xxx.xxx.24.120:13’
set protocols bgp 65151 address-family l2vpn-evpn vni 10013 route-target both ‘1:13’
set protocols bgp 65151 address-family l2vpn-evpn vni 10013 route-target export ‘1:13’
set protocols bgp 65151 address-family l2vpn-evpn vni 10013 route-target import ‘1:13’
set protocols bgp 65151 neighbor xxx.xxx.67.0 peer-group ‘vxlan’
set protocols bgp 65151 neighbor xxx.xxx.67.252 peer-group ‘vxlan’
set protocols bgp 65151 parameters log-neighbor-changes
set protocols bgp 65151 peer-group vxlan address-family l2vpn-evpn soft-reconfiguration inbound
set protocols bgp 65151 peer-group vxlan ebgp-multihop ‘20’
set protocols bgp 65151 peer-group vxlan remote-as ‘6876’
set protocols bgp 65151 peer-group vxlan update-source ‘dum0’
set protocols static route xxx.xxx.0.0/0 next-hop xxx.xxx.24.210
set protocols static route xxx.xxx.24.0/23 next-hop xxx.xxx.24.210
set service snmp community stat authorization ‘ro’
set service snmp community stat network ‘xxx.xxx.0.0/16’
set service snmp location ‘VM-dca’
set service ssh
set system config-management commit-revisions ‘100’
set system console device ttyS0 speed ‘115200’
set system host-name ‘vyos’
set system login user vyos authentication encrypted-password xxxxxx
set system login user vyos authentication plaintext-password xxxxxx
set system name-server ‘xxx.xxx.78.30’
set system name-server ‘xxx.xxx.78.31’
set system ntp server xxx.xxx.80.35
set system syslog global facility all level ‘info’
set system syslog global facility protocols level ‘debug’
set system time-zone ‘Europe/Kiev’
[edit]

vyos@vyos# run show evpn mac vni all

VNI 10012 #MACs (local and remote) 55

Flags: N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy
MAC Type Flags Intf/Remote ES/VTEP VLAN Seq #'s
54:52:94:5b:e2:3d remote 172.20.25.191 0/0
34:0a:98:2f:ef:b1 remote 172.20.24.14 0/0
52:54:e2:f7:5c:a0 remote 172.20.24.103 0/238
52:54:37:bf:9e:a0 remote 172.20.24.100 0/0
00:00:5e:00:01:0c remote 172.20.24.14 0/0
52:54:01:27:f4:6f remote 172.20.24.101 0/0
52:54:0b:9e:66:2d remote 172.20.24.101 0/0
52:54:d2:e3:39:55 remote 172.20.24.100 0/0
b6:80:6c:32:b1:54 local br12 1 0/0
3a:0c:3f:f9:53:c0 remote 172.20.24.121 0/239
52:54:14:69:f2:36 remote 172.20.24.101 0/0
52:54:01:55:ee:17 remote 172.20.24.100 0/2
52:54:a7:f4:e1:ba remote 172.20.24.102 0/0
34:0a:98:2f:ef:f1 remote 172.20.24.6 0/0
52:54:88:75:ca:55 remote 172.20.24.100 0/0
52:54:2a:3e:17:06 remote 172.20.24.100 0/1
32:b1:02:b2:d8:54 remote 172.20.24.103 0/0
52:54:09:3d:45:b4 remote 172.20.24.101 0/1
52:54:15:99:8c:90 remote 172.20.24.103 0/0
52:54:c7:f4:66:49 remote 172.20.24.103 0/0
8a:b7:c1:d0:f2:95 remote 172.20.24.101 0/0
52:54:13:ba:0f:88 remote 172.20.24.103 0/0
52:54:ce:51:38:12 remote 172.20.24.101 0/0
52:54:46:b4:93:97 remote 172.20.24.103 0/0
52:54:95:13:60:4a remote 172.20.24.100 0/1
52:54:61:63:68:3c remote 172.20.24.100 0/0
52:54:cb:d4:20:f5 remote 172.20.24.103 0/0
8a:7c:12:ae:46:27 remote 172.20.24.102 0/1
52:54:20:63:6c:c1 remote 172.20.24.100 0/1
52:54:c1:b8:65:13 remote 172.20.24.101 0/0
52:54:be:a7:20:12 remote 172.20.24.100 0/0
52:54:ea:22:57:93 remote 172.20.24.100 0/0
52:54:1c:7d:91:37 remote 172.20.24.100 0/1
52:54:f7:b2:8a:a2 remote 172.20.24.100 0/0
92:43:ad:76:38:59 remote 172.20.24.100 0/1
52:54:2d:c6:a3:79 remote 172.20.24.102 0/0
8a:b6:6b:e6:98:76 remote 172.20.24.102 0/0
52:54:29:45:6e:3b remote 172.20.25.190 0/1
52:54:21:4c:2b:13 remote 172.20.24.100 0/0
52:54:ec:c6:fe:d8 remote 172.20.24.102 0/0
52:54:24:d8:6a:86 remote 172.20.24.103 0/0
52:54:a0:d4:20:aa remote 172.20.24.101 0/0
52:54:5a:0a:0e:cf remote 172.20.24.102 0/0
52:54:27:05:15:24 remote 172.20.24.101 0/0
52:54:57:2f:cb:db remote 172.20.24.100 0/0
52:54:43:ed:ad:3a remote 172.20.24.101 0/1
52:54:dd:5c:7a:ad remote 172.20.24.103 0/0
52:54:a2:50:ba:e9 remote 172.20.24.100 0/0
52:54:96:8c:fc:2d remote 172.20.24.100 0/0
52:54:36:7d:a7:2d remote 172.20.24.103 0/0
52:54:ae:d2:c4:4d remote 172.20.24.103 0/0
52:54:59:19:66:26 remote 172.20.24.101 0/0
52:54:7b:e8:db:5f remote 172.20.24.103 0/0
52:54:a5:77:97:68 remote 172.20.24.100 0/0
52:54:cf:d9:02:3d remote 172.20.24.100 0/0

VNI 10013 #MACs (local and remote) 14

Flags: N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy
MAC Type Flags Intf/Remote ES/VTEP VLAN Seq #'s
54:52:0d:75:22:3d remote 172.20.25.190 0/0
34:0a:98:2f:ef:b1 remote 172.20.24.14 0/0
54:52:26:78:73:a0 remote 172.20.25.190 0/0
a6:81:16:52:3b:ee local eth1.13 0/0
34:0a:98:2f:ef:f1 remote 172.20.24.6 0/0
54:52:64:06:0e:fd remote 172.20.25.190 0/0
54:52:25:d0:a8:5e remote 172.20.25.190 0/0
00:00:5e:00:01:0d remote 172.20.24.14 0/0
54:52:76:b7:47:91 remote 172.20.25.191 0/0
3e:58:b9:34:ce:89 local br13 1 0/0
54:52:a2:32:5d:c1 remote 172.20.25.190 0/0
54:52:c2:ee:00:7d remote 172.20.25.179 0/0
54:52:b6:5a:db:86 remote 172.20.25.190 0/0
54:52:cc:57:dd:b0 remote 172.20.25.190 0/0
[edit]
vyos@vyos#

I have two vxlanes on the test circuit. vni 10012 has the same problem with arp package.
other routers are Huawei and CumulusVX. they work well. VM-s connected to CumulusVX working well and arp-packet from they coming to VM connected to VyOS also.

I think the problem with the arp-drop is related to the fact that the arp of the source is a router. It schemes usually used at distributed L3 gateway scheme VXLAN. But I don’t see a command which I can disable this feech.

Can you try to add a flood for the VTEPs in question?

bridge fdb append 00:00:00:00:00:00 dst 203.0.113.19 dev vxlan10512

there is yet no CLI available for this

1 Like

i did what you recommended but it didn’t help.

it is clear that there are no problems with the “flood”. also the remote VTEP sends an “arp reply” and this can be seen in the dump on the incoming interface of the vyos. But then this arp reply does not get into the vxlan interface.