Update: Part 2 is live!
I wrote this post for people that may be interested in a scalable Multipoint VPN solution for site-to-site communication. I intend for this to be a multi-part series, so let me know if there’s anything specific you’d like to see related to a deployment like this.
Cool post, thanks!
Waiting for the new posts 
Hi @L0crian,
Thank you for sharing your insightful post on the scalable Multipoint VPN solution for site-to-site communication. I’m eagerly await the subsequent parts of your series! 
Since ZeroTier’s install method is executing a bash script that downloads and installs a .deb package, I’d like to see a method of version persistence since as soon you as you install a new image ZeroTier is no longer installed. Additionally, ZeroTier handles updates through the distro package manager after install so with VyOS there isn’t a clean way, at least with my initial thoughts on it, of handling updates to the software.
root@vyos:/var/lib/zerotier-one# cat >devicemap
# Example: <networkID>=<interface>
xxxxxxxxxxxxxxxx=eth10
Is that persistent across reboots?
Very good write up!
Any custome packages/configs are not persist.
The best way to manage this is to create a symlink to the /config folder. You will need to reinstall ZeroTier when installing a new image.
You can also package ZeroTier with VyOS when building, which is what I do. You still need to recreate the symlinks after a new image install, but you can add that to a post-script.
Example:
cd /var/lib
mv /var/lib/zerotier-one /config/scripts/
ln -s /config/scripts/zerotier-one
The Node Address and related local config are stored in that folder, so the symlink will allow for a persistent deployment.
After I finish the MPLS blog, I’ll make a short one demoing what I mentioned above.
Part 2 is live!
First of all thank you so much for the blog.
I was thinking, isn’t it possible to install zerotier as a container in VyOS thus keeping it alive when installing new VyOS image ? and wouldn’t it be safer this way since there is no alternation on the vyos system directly. ?
really good blog and admirable work on those technologies , there is a trick highlight with the name DMVPN but interesting use case with Zerotier and MPLS ,Moreover, it has the potential to bring forth advanced advantages for the community, encouraging greater utilization of MPLS in enterprise settings. . @JoeN @syncer
@a.katib91, it’s actually funny you mention that, I had the same thought yesterday and labbed it real quick; works fiine! The next post will be a quick one showing both the symlink and container methods of keeping the ZeroTier deployment persistent across image upgrades.
I’m also running ZeroTier on VyOS for site to site with OSPF, but I prefer to use containers. This is what that config looks like:
container {
name zt {
allow-host-networks
cap-add sys-admin
cap-add net-admin
device tun {
destination /dev/net/tun
source /dev/net/tun
}
image docker.io/zyclonite/zerotier:latest
volume data {
destination /var/lib/zerotier-one
source /config/zerotier-one
}
}
}
My /config/zerotier-one directory contains the zerotier files like identity.public, identity.secret, networks.d/ and so on and which survives image upgrades and with no need for installation of a deb package on the system.
A few other hacks include setting a custom /config/zerotier-one/networks.d/xxxxxxxxxxxxxx.local.conf file which makes sure that zerotier only assigns the IP on the zt adapter and does not inject any routes (since I want OSPF to manage that):
allowManaged=192.168.195.44/24
allowGlobal=0
allowDefault=0
And also adding a mapping to /config/zerotier-one/devicemap which just renames the ztxxxxx adapter name to e.g. tun195 so that it shows up with show interfaces:
xxxxxxxxxxxxxx=tun195
You can enter the running container with connect container zt to work with the zerotier-cli.
I can’t seem to edit my original post any more.
I uploaded a part 1.5 for the series about persistence for the ZeroTier deployment:
Thanks! I’m familiar with persistence methods personally but figured I should point it out in case other people follow the guide and then run into that issue.
Thanks for bringing it up, it was definitely a good addition.
Thanks for the excellent tutorial and writeup @L0crian, really good stuff!
FWIW, I’m just adding my experience here, maybe it’s useful for someone.
Before container support in VyOS and since I wanted to restrict the ZeroTier installation to /config/, there was another variant - copying the zerotier-one binary from another Debian system into /config/zerotier-one/ and adding a few symlinks:
pushd /config/zerotier-one
ln -s zerotier-one zerotier-cli
ln -s zerotier-one zerotier-idtool
echo export ZEROTIER_HOME="/config/zerotier-one" >> $HOME/.profile
echo export PATH='$ZEROTIER_HOME:$PATH' >> $HOME/.profile
Together with a systemd service file.
ahovda@oslfirewall:~$ cat /config/zerotier-one/zerotier-one.service
[Unit]
Description=ZeroTier One
After=network-online.target
[Service]
Environment=ZEROTIER_HOME='/config/zerotier-one'
ExecStart=/config/zerotier-one/zerotier-one -U
AmbientCapabilities=CAP_NET_RAW CAP_NET_ADMIN
User=ahovda
Group=vyattacfg
[Install]
WantedBy=multi-user.target
And then register and start it on every boot:
ahovda@oslfirewall:~$ cat /config/scripts/vyatta-postconfig-bootup.script
#!/bin/sh
systemctl enable --now /config/zerotier-one/zerotier-one.service
Taken from memory, but it used to work fine. I agree the container version is still superior. 
Added a post about increasing the scale of this design:
Added another post to this design: Microsegmentation
Next level stuff, love it. Keep it coming!
I plan for this to be the final post in this series unless there’s something specific someone wants to see.
@L0crian Once again, a big thank you for creating this series! Your insights have been truly valuable. We appreciate your contributions, and I hope to see more from you in the future. 
Cheers!