What is the best vpn protocol to use for site to site on vyos?

If you want something that scale then dynamic routing compatibility is really mandatory.

DMVPN should work, and according to this thread:
https://forum.vyos.io/t/can-wireguard-replace-dmvpn/3219

could be a better choice than WireGuard for mesh deployment, but since I have no experience with it I will let more knowledgeable people answer.