Hey folks, I’m developing a lab environment to learn BGP in VyOS, starting with the goal of initiating a simple peering session and then incrementally adding more complex scenarios modeled after both ISP and enterprise networks. However, I’m having trouble getting the session up and advertising routes across from both sides.
To begin, I’m using two 1.4-Rolling VyOS VMs running within Proxmox on the same bridge, with the following configurations:
vyos@vyos-r1:~$ show configuration
interfaces {
ethernet eth0 {
address dhcp
hw-id 00:50:56:7d:31:d5
}
ethernet eth1 {
address 10.0.0.1/30
hw-id 00:50:56:33:d2:19
}
ethernet eth2 {
hw-id 00:50:56:87:a0:20
}
ethernet eth3 {
hw-id 00:50:56:69:73:87
}
ethernet eth4 {
hw-id 00:50:56:14:a8:ab
}
loopback lo {
}
}
protocols {
bgp {
address-family {
ipv4-unicast {
network 10.1.0.0/16 {
}
}
}
neighbor 10.0.0.2 {
ebgp-multihop 2
remote-as 65535
update-source 10.0.0.1
}
parameters {
router-id 10.0.0.1
}
system-as 65534
}
static {
route 10.1.0.0/16 {
blackhole {
distance 254
}
}
}
}
service {
ntp {
allow-client {
address 0.0.0.0/0
address ::/0
}
server time1.vyos.net {
}
server time2.vyos.net {
}
server time3.vyos.net {
}
}
ssh {
}
}
system {
config-management {
commit-revisions 100
}
conntrack {
modules {
ftp
h323
nfs
pptp
sip
sqlnet
tftp
}
}
console {
device ttyS0 {
speed 115200
}
}
host-name vyos-r1
login {
user vyos {
authentication {
encrypted-password ****************
plaintext-password ****************
}
}
}
syslog {
global {
facility all {
level info
}
facility local7 {
level debug
}
}
}
}
vyos@vyos-r2:~$ show configuration
interfaces {
ethernet eth0 {
address dhcp
hw-id 00:50:56:7b:cb:aa
}
ethernet eth1 {
address 10.0.0.2/30
hw-id 00:50:56:fd:86:22
}
ethernet eth2 {
hw-id 00:50:56:43:e8:46
}
ethernet eth3 {
hw-id 00:50:56:f4:4c:55
}
ethernet eth4 {
hw-id 00:50:56:b6:fe:ad
}
loopback lo {
}
}
protocols {
bgp {
address-family {
ipv4-unicast {
network 10.2.0.0/16 {
}
}
}
neighbor 10.0.0.1 {
ebgp-multihop 2
remote-as 65534
update-source 10.0.0.2
}
parameters {
router-id 10.0.0.2
}
system-as 65535
}
static {
route 10.1.0.0/16 {
blackhole {
distance 254
}
}
}
}
service {
ntp {
allow-client {
address 0.0.0.0/0
address ::/0
}
server time1.vyos.net {
}
server time2.vyos.net {
}
server time3.vyos.net {
}
}
ssh {
}
}
system {
config-management {
commit-revisions 100
}
conntrack {
modules {
ftp
h323
nfs
pptp
sip
sqlnet
tftp
}
}
console {
device ttyS0 {
speed 115200
}
}
host-name vyos-r2
login {
user vyos {
authentication {
encrypted-password ****************
plaintext-password ****************
}
}
}
syslog {
global {
facility all {
level info
}
facility local7 {
level debug
}
}
}
}
R1 reports the following on its BGP status:
vyos@vyos-r1:~$ show bgp summary
% No BGP neighbors found in VRF default
vyos@vyos-r1:~$ show bgp ipv4
BGP table version is 3, local router ID is 10.0.0.1, vrf id 0
Default local pref 100, local AS 65534
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> 10.1.0.0/16 0.0.0.0 0 32768 i
Displayed 1 routes and 1 total paths
vyos@vyos-r1:~$ show bgp neighbors
BGP neighbor is 10.0.0.2, remote AS 65535, local AS 65534, external link
Local Role: undefined
Remote Role: undefined
BGP version 4, remote router ID 0.0.0.0, local router ID 10.0.0.1
BGP state = Idle
Last read 00:35:20, Last write never
Hold time is 180 seconds, keepalive interval is 60 seconds
Configured hold time is 180 seconds, keepalive interval is 60 seconds
Configured conditional advertisements interval is 60 seconds
Graceful restart information:
Local GR Mode: Helper*
Remote GR Mode: NotApplicable
R bit: False
N bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 0
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 0 0
Notifications: 0 0
Updates: 0 0
Keepalives: 0 0
Route Refresh: 0 0
Capability: 0 0
Total: 0 0
Minimum time between advertisement runs is 0 seconds
Update source is 10.0.0.1
Connections established 0; dropped 0
Last reset 00:35:20, No AFI/SAFI activated for peer
External BGP neighbor may be up to 2 hops away.
BGP Connect Retry Timer in Seconds: 120
Read thread: off Write thread: off FD used: -1
R2 reports the following on its BGP status:
vyos@vyos-r2:~$ show bgp summary
% No BGP neighbors found in VRF default
vyos@vyos-r2:~$ show bgp ipv4
BGP table version is 3, local router ID is 10.0.0.2, vrf id 0
Default local pref 100, local AS 65535
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> 10.2.0.0/16 0.0.0.0 0 32768 i
Displayed 1 routes and 1 total paths
vyos@vyos-r2:~$ show bgp neighbors
BGP neighbor is 10.0.0.1, remote AS 65534, local AS 65535, external link
Local Role: undefined
Remote Role: undefined
BGP version 4, remote router ID 0.0.0.0, local router ID 10.0.0.2
BGP state = Idle
Last read 00:36:51, Last write never
Hold time is 180 seconds, keepalive interval is 60 seconds
Configured hold time is 180 seconds, keepalive interval is 60 seconds
Configured conditional advertisements interval is 60 seconds
Graceful restart information:
Local GR Mode: Helper*
Remote GR Mode: NotApplicable
R bit: False
N bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 0
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 0 0
Notifications: 0 0
Updates: 0 0
Keepalives: 0 0
Route Refresh: 0 0
Capability: 0 0
Total: 0 0
Minimum time between advertisement runs is 0 seconds
Update source is 10.0.0.2
Connections established 0; dropped 0
Last reset 00:36:51, No AFI/SAFI activated for peer
External BGP neighbor may be up to 2 hops away.
BGP Connect Retry Timer in Seconds: 120
Read thread: off Write thread: off FD used: -1
So no good on either side. However, I can ping back and forth between the two VMs over the eth1 interfaces I’m using to peer:
vyos@vyos-r1:~$ ping 10.0.0.2 interface eth1 count 4
PING 10.0.0.2 (10.0.0.2) from 10.0.0.1 eth1: 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.273 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.238 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.274 ms
64 bytes from 10.0.0.2: icmp_seq=4 ttl=64 time=0.189 ms
--- 10.0.0.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3092ms
rtt min/avg/max/mdev = 0.189/0.243/0.274/0.034 ms
vyos@vyos-r2:~$ monitor traffic interface eth1
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:14:02.695849 IP 10.0.0.1 > 10.0.0.2: ICMP echo request, id 25295, seq 1, length 64
16:14:02.695886 IP 10.0.0.2 > 10.0.0.1: ICMP echo reply, id 25295, seq 1, length 64
16:14:03.740202 IP 10.0.0.1 > 10.0.0.2: ICMP echo request, id 25295, seq 2, length 64
16:14:03.740239 IP 10.0.0.2 > 10.0.0.1: ICMP echo reply, id 25295, seq 2, length 64
16:14:04.764254 IP 10.0.0.1 > 10.0.0.2: ICMP echo request, id 25295, seq 3, length 64
16:14:04.764277 IP 10.0.0.2 > 10.0.0.1: ICMP echo reply, id 25295, seq 3, length 64
16:14:05.788187 IP 10.0.0.1 > 10.0.0.2: ICMP echo request, id 25295, seq 4, length 64
16:14:05.788210 IP 10.0.0.2 > 10.0.0.1: ICMP echo reply, id 25295, seq 4, length 64
16:14:07.752411 ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 28
16:14:07.752610 ARP, Reply 10.0.0.1 is-at 00:50:56:33:d2:19 (oui Unknown), length 28
16:14:07.900182 ARP, Request who-has 10.0.0.2 tell 10.0.0.1, length 28
16:14:07.900192 ARP, Reply 10.0.0.2 is-at 00:50:56:fd:86:22 (oui Unknown), length 28
^C
12 packets captured
12 packets received by filter
0 packets dropped by kernel
And BGP is up and listening on both interfaces:
vyos@vyos-r1:~$ nmap 10.0.0.2
Starting Nmap 7.93 ( https://nmap.org ) at 2023-07-27 16:17 UTC
Nmap scan report for 10.0.0.2
Host is up (0.00029s latency).
Not shown: 998 closed tcp ports (conn-refused)
PORT STATE SERVICE
22/tcp open ssh
179/tcp open bgp
Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds
vyos@vyos-r2:~$ nmap 10.0.0.1
Starting Nmap 7.93 ( https://nmap.org ) at 2023-07-27 16:18 UTC
Nmap scan report for 10.0.0.1
Host is up (0.00027s latency).
Not shown: 998 closed tcp ports (conn-refused)
PORT STATE SERVICE
22/tcp open ssh
179/tcp open bgp
Nmap done: 1 IP address (1 host up) scanned in 0.10 seconds
I though I did everything correct, but I must be missing something. It doesn’t seem that a prefix-list is necessary, and defaults to permit in that case.
I’m hoping to develop a set of tutorials on learning BGP from the ground up (along with routing in general) using VyOS, so any help and best practices advice is much appreciated!
