ARTICLE: Using NetBird for Site-to-Site Routing on VyOS

Guides & HowTo
1

UPDATE: Part 2 with an Exit Node is available!

I previously did a few blog posts on using NetBird with VyOS, but those were related to more advanced topics, like using VxLAN with EVPN over NetBird, allowing for secure site-to-site multitenant traffic. You can find those here:
Scalable and Secure VxLAN Multisite using NetBird – Part 1:Initial Config-L3VPN
Scalable and Secure VxLAN Multisite using NetBird – Part 2: L2VPN

This Article focuses on using NetBird as more of a closed solution for a multisite VPN solution (no additional routing protocols; all routing handled by NetBird).

Part 2 with Exit Node:

5 Likes
2

Hii

first of all, thankkk you soo much for your tutorials and posts. honestly amazing stuff and clearly laid out for me to learn so much.

using netbird, I have been trying to have 2 vyos routers where one acts as an exit node, and the other to route traffic from certain servers on lan through the exit node. but I just can’t seem to make it work. would you try that and if you have time make a tutorial on how to do it properly ? many thanks in advance.

1 Like
3

I’m glad you’re liking the posts. I just tried it to make sure there wasn’t anything odd with doing it on VyOS and it works fine. I’ll do a quick blog post on it. I’ll probably publish it later tomorrow, or early Saturday.

4

awesome. basically the problem i’m facing when I advertise the exit node on vyos#1 to vyos#2, it automatically route all of vyos#2 through it, whereas I want to do policy based routing in vyos#2 for few devices only on lan and keep the main separate.

I tried to remove ‘allow-host-networking’ and put in the ip-address manually and did other tests but wasn’t successful. probably something i’m missing in the way netbird managing the routes.

5

@a.katib91
Part 2 is live:
Using NetBird for Site-to-Site Routing on VyOS – Exit Node

What you’re trying to do is a little different though, since I just showed a simple solution. NetBird already uses PBR to forward its traffic into table 7120, so you’re trying to do PBR on top of PBR. You can see the rule (rule 110) with ip rule show.

vyos@NB-LAN# sudo ip rule show
0:      from all lookup local
100:    from all lookup main suppress_prefixlength 0
110:    not from all fwmark 0x1bd00 lookup 7120
32766:  from all lookup main
32767:  from all lookup default

You can see that traffic that does not have a mark of 0x1bd00 will be sent to the NetBird table (7120).

So the simple solution is to just mark the traffic that you want to use the local internet of vyos#2 with that mark. 0x1bd00 maps to a decimal value of 113920. So we create a policy route rule for our traffic. This will prevent 10.0.1.10 from using NetBird’s PBR.

set policy route TEST interface 'eth1'
set policy route TEST rule 10 action 'accept'
set policy route TEST rule 10 set mark '113920'
set policy route TEST rule 10 source address '10.0.1.10'

And we test:

Traceroute from 10.0.1.10 to 4.2.2.2:
10_0_1_10> trace 4.2.2.2 -P 1 -m 15
trace to 4.2.2.2, 15 hops max (ICMP), press Ctrl+C to stop
 1   10.0.1.1   0.720 ms  0.611 ms  0.638 ms
 2   10.0.95.1   4.084 ms  4.156 ms  5.145 ms
 3   172.16.0.1   2.620 ms  2.580 ms  3.695 ms
 4   .... hops omitted
 9   4.2.2.2   8.153 ms  10.280 ms  11.141 ms

Traceroute from 10.0.1.11 to 4.2.2.2:
10_0_1_11> trace 4.2.2.2 -P 1 -m 15
trace to 4.2.2.2, 15 hops max (ICMP), press Ctrl+C to stop
 1   10.0.1.1   0.834 ms  0.659 ms  0.671 ms
 2   100.90.34.56   2.499 ms  2.394 ms  2.371 ms
 3   10.0.95.1   8.439 ms  5.915 ms  5.295 ms
 4   172.16.0.1   4.212 ms  3.877 ms  4.276 ms
 5   ... hops omitted
10   4.2.2.2   17.476 ms  16.037 ms  15.031 ms

You can see that 10.0.1.10 takes the local internet path (the 10.0.95.1 hop), and 10.0.1.11 will take the NetBird path since we aren’t matching that in our policy route rule.

1 Like
6

Thank you soo much amazing. not only you addressed the problem but explained very well how to troubleshoot the issue (which is more important in my opinion). honestly, you’re the best out there to make these tutorials period.

1 Like