Having several internet connections, requires more configuration that you might think.
You must ensure that if incoming traffic is received from internet, for example in eth0-ISP, then the response should also be routed through same interface. Otherwise, the response will have another source ip address (let say eth4-ISPX public IP address), and client who initiated the connection will not accept this response, since it comes from a different IP.
This can be achieved, but as far as I remember, not directly from vyos cli. An idea on how to achieve this? You can try WAN Load Balancing, and enable sticky-connections. Then analize firewall rules and policies.
Also, a thread with similar requirements can be found here