VxLAN with VRF commit traceback bug?

Bugs
1

Hi All,

Trying to do a Layer2 VxLAN extension with EVPN control plane in Vyos, but with multi-tenancy (VRF’s). When I try and apply the below command (which I understand is needed to share the MACs into EVPN) I get a commit traceback failure.

Please can you assist if this is a bug, or a config / user error?

vyos@vyos-a# set vrf name BLUE protocols bgp address-family l2vpn-evpn advertise-all-vni
[edit]

vyos@vyos-a# commit
[ vrf name BLUE protocols bgp ]
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):

  • Contact us using the online help desk if you have a subscription:

  • Make sure you are running the latest version of VyOS available at:

  • Consult the community forum to see how to handle this issue:

  • Join us on Slack where our users exchange help and advice:

When reporting problems, please include as much information as possible:

  • do not obfuscate any data (feel free to contact us privately if your
    business policy requires it)
  • and include all the information presented below

Report time: 2024-04-18 13:03:47
Image version: VyOS 1.4-rolling-202402250401
Release train: sagitta

Built by: root@5707fe13260c
Built on: Sun 25 Feb 2024 04:01 UTC
Build UUID: 61ebf4fb-61db-4140-b588-e0ec3abd37f0
Build commit ID: b5d545836916db-dirty

Architecture: x86_64
Boot via: installed image
System type: VMware guest

Hardware vendor: VMware, Inc.
Hardware model: VMware Virtual Platform
Hardware S/N: VMware-42 25 39 f0 84 ed 1c 21-c3 bb 95 73 80 7c 52 a4
Hardware UUID: f0392542-ed84-211c-c3bb-9573807c52a4

Traceback (most recent call last):
File “/usr/libexec/vyos/conf_mode/protocols_bgp.py”, line 611, in
apply(c)
File “/usr/libexec/vyos/conf_mode/protocols_bgp.py”, line 602, in apply
frr_cfg.commit_configuration(bgp_daemon)
File “/usr/lib/python3/dist-packages/vyos/frr.py”, line 482, in commit_configuration
raise ConfigurationNotValid(f’Config commit retry counter ({count_max}) exceeded for {daemon} dameon!')
vyos.frr.ConfigurationNotValid: Config commit retry counter (5) exceeded for bgpd dameon!

[[vrf name BLUE protocols bgp]] failed
Commit failed
[edit]
vyos@vyos-a#

Below is my bridge / VRF config:-

set interfaces bridge br1000 address ‘172.16.1.100/24’
set interfaces bridge br1000 mac ‘00:50:56:a5:92:6c’
set interfaces bridge br1000 member interface eth2.1000
set interfaces bridge br1000 member interface vxlan1000
set interfaces bridge br1000 vrf ‘BLUE’

set interfaces vxlan vxlan1000 description ‘customer blue’
set interfaces vxlan vxlan1000 parameters nolearning
set interfaces vxlan vxlan1000 port ‘4789’
set interfaces vxlan vxlan1000 source-address ‘10.1.1.3’
set interfaces vxlan vxlan1000 vni ‘1000’

set vrf name BLUE protocols bgp address-family ipv4-unicast redistribute connected
set vrf name BLUE protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name BLUE protocols bgp address-family l2vpn-evpn advertise-svi-ip
set vrf name BLUE protocols bgp address-family l2vpn-evpn rt-auto-derive
set vrf name BLUE protocols bgp address-family l2vpn-evpn vni 1000 advertise-svi-ip
set vrf name BLUE protocols bgp system-as ‘65100’
set vrf name BLUE table ‘1000’
set vrf name BLUE vni ‘1000’

2

it looks like an error in your enviorment, I’ve tested it on a VM ,it works without problems :

vyos@vyos:~$ show configuration commands  | match 'vrf|inter'
set interfaces bridge br1000 address '172.16.1.100/24'
set interfaces bridge br1000 member interface eth2
set interfaces bridge br1000 member interface eth3
set interfaces bridge br1000 vrf 'BLUE'
set interfaces ethernet eth0 hw-id '50:00:00:01:00:00'
set interfaces ethernet eth1 address '10.1.1.3/24'
set interfaces ethernet eth1 hw-id '50:00:00:01:00:01'
set interfaces ethernet eth2 hw-id '50:00:00:01:00:02'
set interfaces ethernet eth3 hw-id '50:00:00:01:00:03'
set interfaces loopback lo
set interfaces vxlan vxlan1000 description 'customer blue'
set interfaces vxlan vxlan1000 parameters nolearning
set interfaces vxlan vxlan1000 port '4789'
set interfaces vxlan vxlan1000 source-address '10.1.1.3'
set interfaces vxlan vxlan1000 vni '1000'
set vrf name BLUE protocols bgp address-family ipv4-unicast redistribute connected
set vrf name BLUE protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set vrf name BLUE protocols bgp address-family l2vpn-evpn advertise-all-vni
set vrf name BLUE protocols bgp address-family l2vpn-evpn advertise-svi-ip
set vrf name BLUE protocols bgp address-family l2vpn-evpn rt-auto-derive
set vrf name BLUE protocols bgp address-family l2vpn-evpn vni 1000 advertise-svi-ip
set vrf name BLUE protocols bgp system-as '65100'
set vrf name BLUE table '1000'
set vrf name BLUE vni '1000'
3

I upgraded the latest 1.5, but still I can’t get it to commit. This same error happens in both a Vmware & Azure enviroments. This time… the traceback gave a little more information about the error:-

vyos@vyos-a# commit

[5129|mgmtd] sending configuration [5130|zebra] sending configuration
[5131|ripd] sending configuration [5132|ripngd] sending configuration
[5134|ospf6d] sending configuration [5133|ospfd] sending configuration
[5135|ldpd] sending configuration [5136|bgpd] sending configuration
[5137|isisd] sending configuration [5141|babeld] sending configuration
[5130|zebra] done [5132|ripngd] done [5129|mgmtd] done [5144|watchfrr]
sending configuration [5131|ripd] done [5133|ospfd] done [5134|ospf6d]
done [5137|isisd] done [5146|staticd] sending configuration [5135|ldpd]
done % Please unconfigure EVPN in VRF default line 7: Failure to
communicate[13] to bgpd, line: advertise-all-vni [5141|babeld] done
[5147|bfdd] sending configuration This command is only supported under
EVPN VRF [5144|watchfrr] done [5136|bgpd] Configuration
file[/etc/frr/frr.conf] processing failure: 13 Waiting for children to
finish applying config… [5150|pim6d] sending configuration [5147|bfdd]
done [5146|staticd] done [5150|pim6d] done [5154|mgmtd] sending
configuration [5155|zebra] sending configuration [5156|ripd] sending
configuration [5157|ripngd] sending configuration [5159|ospf6d] sending
configuration [5160|ldpd] sending configuration [5158|ospfd] sending
configuration [5161|bgpd] sending configuration [5162|isisd] sending
configuration [5166|babeld] sending configuration [5155|zebra] done
[5154|mgmtd] done [5157|ripngd] done [5156|ripd] done [5158|ospfd] done
[5169|watchfrr] sending configuration [5172|bfdd] sending configuration
[5171|staticd] sending configuration [5160|ldpd] done [5159|ospf6d] done
[5162|isisd] done [5166|babeld] done % Please unconfigure EVPN in VRF
default line 7: Failure to communicate[13] to bgpd, line: advertise-
all-vni Waiting for children to finish applying config… [5175|pim6d]
sending configuration This command is only supported under EVPN VRF %
Please unconfigure EVPN in VRF default line 23: Failure to
communicate[13] to bgpd, line: advertise-all-vni This command is only
supported under EVPN VRF [5161|bgpd] Configuration
file[/etc/frr/frr.conf] processing failure: 13 [5169|watchfrr] done
[5172|bfdd] done [5171|staticd] done [5175|pim6d] done

[[vrf name BLUE protocols bgp]] failed
Commit failed
[edit]

4

This issue has seemed to be in FRR for a while now:

I just tested it, and your config pastes in fine if you don’t have any global BGP config, which I’m assuming is how fernando tested as well. If you also have global BGP config, then it does the Please unconfigure EVPN in VRF default error in FRR. This happens in when configuring within VyOS or FRR.

Reading through the PR where they added the feature to FRR, it looks like they did checks to ensure advertise-all-vni could only exists in one location. The check must incorrectly be flagging that the advertise-all-vni exists elsewhere.