I ran into a bizarre issue when trying to change a neighbor’s peer group that resulted in errors + making FRR think that the neighbor had no remote-as defined. The neighbor was set to shutdown before proceeding with any of these changes. I ended up having to reboot the router in order to get that neighbor’s session back up.
First, I verified the neighbors sessions were shutdown. I then proceeded to set the peer-group to a different peer-group (downstream) from what it was previously (ix).
The following error was displayed, however the changes seemed to apply and was comitted.
% Cannot change the peer-group. Deconfigure first
Error configuring routing subsystem. See log for more detailed information
My interpretation of this was that the peer-group needed to be removed first, so I proceeded to remove the peer-group entirely from this neighbor. Same message.
% Specify remote-as or peer-group commands first
Error configuring routing subsystem. See log for more detailed information
I then re-applied the downstream peer-group, ignored the error message and turned the neighbors session back up by removing “shutdown” on the neighbor. The session failed to establish as it looks like FRR had a remote-as of 0 even though it was defined in the config.
My questions are as follows:
- Do I need to entirely remove a neighbor, commit, then add the neighbor back to change a peer-group? Is it not possible to change a peer-group on a neighbor without deconfiguring the entire neighbor?
- Why did this cause the remote-as to disappear from FRR even though it was never modified in the config?
Any insight is greatly appreciated!
Full output below:
vyos@edge1.btd1.us.as951.net:~$ show bgp summary
IPv4 Unicast Summary:
BGP router identifier 23.138.8.1, local AS number 951 vrf-id 0
BGP table version 2399391
RIB entries 1672722, using 306 MiB of memory
Peers 16, using 341 KiB of memory
Peer groups 4, using 256 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
23.138.8.22 4 950 1998 467088 0 0 0 1d09h14m 1 910384
2602:fb51:1:2::22 4 950 2000 1088412 0 0 0 1d09h14m NoNeg
206.83.43.9 4 42 2211 2001 0 0 0 1d09h15m 48 2
206.83.43.10 4 3856 1999 2002 0 0 0 1d09h15m 1 2
206.83.43.30 4 6939 59368 2001 0 0 0 1d09h15m 137172 2
206.83.43.46 4 394414 10279 2005 0 0 0 00:03:16 Idle (Admin) 0
2001:504:9b::9 4 42 2205 2004 0 0 0 1d09h15m NoNeg
2001:504:9b::10 4 3856 1999 2005 0 0 0 1d09h15m NoNeg
2001:504:9b::46 4 394414 10049 2005 0 0 0 00:03:16 Idle (Admin) 0
206.83.43.1 4 397806 149110 2002 0 0 0 1d09h15m 98563 2
206.83.43.2 4 397806 149089 2008 0 0 0 1d09h15m 98821 2
2001:504:9b::1 4 397806 625673 2004 0 32 0 1d09h15m NoNeg
2001:504:9b::2 4 397806 625809 2020 0 31 0 1d09h15m NoNeg
192.190.136.105 4 62943 508655 3997 0 24 0 1d09h15m 910277 2
2001:504:9b::30 4 6939 856934 2001 0 164 0 1d09h15m NoNeg
2604:2980:f00::5 4 62943 255872 3997 0 10 0 1d09h15m NoNeg
Total number of neighbors 16
IPv6 Unicast Summary:
BGP router identifier 23.138.8.1, local AS number 951 vrf-id 0
BGP table version 2062041
RIB entries 289531, using 53 MiB of memory
Peers 16, using 341 KiB of memory
Peer groups 4, using 256 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
23.138.8.22 4 950 1998 467088 0 0 0 1d09h14m NoNeg
2602:fb51:1:2::22 4 950 2000 1088412 0 0 0 1d09h14m 1 150298
206.83.43.9 4 42 2211 2001 0 0 0 1d09h15m NoNeg
206.83.43.10 4 3856 1999 2002 0 0 0 1d09h15m NoNeg
206.83.43.30 4 6939 59368 2001 0 0 0 1d09h15m NoNeg
206.83.43.46 4 394414 10279 2005 0 0 0 00:03:16 Idle (Admin) 0
2001:504:9b::9 4 42 2205 2004 0 0 0 1d09h15m 23 2
2001:504:9b::10 4 3856 1999 2005 0 0 0 1d09h15m 1 2
2001:504:9b::46 4 394414 10049 2005 0 0 0 00:03:16 Idle (Admin) 0
206.83.43.1 4 397806 149110 2002 0 0 0 1d09h15m NoNeg
206.83.43.2 4 397806 149089 2008 0 0 0 1d09h15m NoNeg
2001:504:9b::1 4 397806 625673 2004 0 32 0 1d09h15m 40351 2
2001:504:9b::2 4 397806 625809 2020 0 31 0 1d09h15m 40424 2
192.190.136.105 4 62943 508655 3997 0 24 0 1d09h15m NoNeg
2001:504:9b::30 4 6939 856934 2001 0 164 0 1d09h15m 148485 2
2604:2980:f00::5 4 62943 255872 3997 0 10 0 1d09h15m 150272 2
Total number of neighbors 16
[edit]
vyos@edge1.btd1.us.as951.net# set protocols bgp 951 neighbor 206.83.43.46 peer-group downstream
[edit]
vyos@edge1.btd1.us.as951.net# set protocols bgp 951 neighbor 2001:504:9b::46 peer-group downstream
[edit]
vyos@edge1.btd1.us.as951.net# commit
[ protocols bgp 951 ]
% Cannot change the peer-group. Deconfigure first
Error configuring routing subsystem. See log for more detailed information
[edit]
vyos@edge1.btd1.us.as951.net# delete protocols bgp 951 neighbor 206.83.43.46 peer-group
[edit]
vyos@edge1.btd1.us.as951.net# delete protocols bgp 951 neighbor 2001:504:9b::46 peer-group
[edit]
vyos@edge1.btd1.us.as951.net# commit
[ protocols bgp 951 ]
% Specify remote-as or peer-group commands first
Error configuring routing subsystem. See log for more detailed information
vyos@edge1.btd1.us.as951.net:~$ show bgp summary
IPv4 Unicast Summary:
BGP router identifier 23.138.8.1, local AS number 951 vrf-id 0
BGP table version 2405363
RIB entries 1672339, using 306 MiB of memory
Peers 16, using 341 KiB of memory
Peer groups 4, using 256 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
23.138.8.22 4 950 2019 469012 0 0 0 1d09h35m 1 910173
206.83.43.46 4 0 1 7 0 0 0 never Active 0
2001:504:9b::46 4 0 0 3 0 0 0 never Active 0
2602:fb51:1:2::22 4 950 2021 1093257 0 0 0 1d09h35m NoNeg
206.83.43.9 4 42 2235 2023 0 0 0 1d09h37m 48 2
206.83.43.10 4 3856 2021 2024 0 0 0 1d09h37m 1 2
206.83.43.30 4 6939 59698 2023 0 0 0 1d09h37m 137066 2
2001:504:9b::9 4 42 2229 2026 0 0 0 1d09h37m NoNeg
2001:504:9b::10 4 3856 2021 2027 0 0 0 1d09h37m NoNeg
206.83.43.1 4 397806 149453 2024 0 0 0 1d09h37m 98454 2
206.83.43.2 4 397806 149442 2032 0 0 0 1d09h37m 98711 2
2001:504:9b::1 4 397806 627062 2026 0 0 0 1d09h37m NoNeg
2001:504:9b::2 4 397806 627198 2042 0 0 0 1d09h37m NoNeg
192.190.136.105 4 62943 510423 4040 0 0 0 1d09h37m 910075 2
2001:504:9b::30 4 6939 861693 2023 0 0 0 1d09h37m NoNeg
2604:2980:f00::5 4 62943 257511 4040 0 0 0 1d09h37m NoNeg
Total number of neighbors 16
IPv6 Unicast Summary:
BGP router identifier 23.138.8.1, local AS number 951 vrf-id 0
BGP table version 2070912
RIB entries 289544, using 53 MiB of memory
Peers 16, using 341 KiB of memory
Peer groups 4, using 256 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
23.138.8.22 4 950 2019 469012 0 0 0 1d09h35m NoNeg
206.83.43.46 4 0 1 7 0 0 0 never Active 0
2001:504:9b::46 4 0 0 3 0 0 0 never Active 0
2602:fb51:1:2::22 4 950 2021 1093257 0 0 0 1d09h35m 1 150307
206.83.43.9 4 42 2235 2023 0 0 0 1d09h37m NoNeg
206.83.43.10 4 3856 2021 2024 0 0 0 1d09h37m NoNeg
206.83.43.30 4 6939 59698 2023 0 0 0 1d09h37m NoNeg
2001:504:9b::9 4 42 2229 2026 0 0 0 1d09h37m 23 2
2001:504:9b::10 4 3856 2021 2027 0 0 0 1d09h37m 1 2
206.83.43.1 4 397806 149453 2024 0 0 0 1d09h37m NoNeg
206.83.43.2 4 397806 149442 2032 0 0 0 1d09h37m NoNeg
2001:504:9b::1 4 397806 627062 2026 0 0 0 1d09h37m 40344 2
2001:504:9b::2 4 397806 627198 2042 0 0 0 1d09h37m 40416 2
192.190.136.105 4 62943 510423 4040 0 0 0 1d09h37m NoNeg
2001:504:9b::30 4 6939 861693 2023 0 0 0 1d09h37m 148494 2
2604:2980:f00::5 4 62943 257511 4040 0 0 0 1d09h37m 150282 2
Total number of neighbors 16
vyos@edge1.btd1.us.as951.net:~$
vyos@edge1.btd1.us.as951.net# show protocols bgp 951 neighbor 206
206.83.43.1 206.83.43.10 206.83.43.2 206.83.43.30 206.83.43.46 206.83.43.9
[edit]
vyos@edge1.btd1.us.as951.net# show protocols bgp 951 neighbor 206.83.43.46
address-family {
ipv4-unicast {
maximum-prefix 1
}
}
description "Peer: E2W Solutions AS394414"
peer-group downstream
remote-as 394414
[edit]
vyos@edge1.btd1.us.as951.net# show protocols bgp 951 neighbor 2001
2001:504:9b::1 2001:504:9b::10 2001:504:9b::2 2001:504:9b::30 2001:504:9b::46 2001:504:9b::9
[edit]
vyos@edge1.btd1.us.as951.net# show protocols bgp 951 neighbor 2001:504:9b::46
address-family {
ipv6-unicast {
maximum-prefix 8
}
}
description "Peer: E2W Solutions AS394414"
peer-group downstream
remote-as 394414
[edit]