Looking for suggestions on firewall for multiple vpn servers

scenario: i built 4 big vyos boxes to be openvpn servers because I need to handle potentially thousands of users.

some users may be using the vpn to access the internet only.
other users might be allowed to access some server resources on the private network
some users use a full vpn tunnel others use split tunneling
some users can access 1194udp and others can only access 443tcp
some users might be edgerouters with a subnet behind them
some users need to communicate over the vpn thru another vpn to a remote site.

so since i want to support both tcp and udp, and because I want some users to be full vpn and some to be split I ended up with 4 openvpn instances on each vyos

vpn1 vtun0: 10.224.0.0/16
vpn1 vtun1: 10.225.0.0/16
vpn1 vtun2: 10.226.0.0/16
vpn1 vtun3: 10.227.0.0/16
vpn2 vtun0: 10.228.0.0/16

so for the users who only need internet access I dont define their user at all so they get a dynamic IP from the subnet and that’s fine, the firewall says !10.0.0.0/8 and no problemo.

for users who are allowed to access specific resources, I’m giving each user an IP in each subnet (since they could theoretically connect to any of the 16 openvpn instances…

lets say user1 is 10.224.254.101 on vpn1 vtun0 and 10.228.254.101 on vpn2 vtun0. and lets say user1 is allowed to access the network 10.10.0.0/20…

so I create a network group on vpn1 that looks like this:

set firewall group network-group user1 network 10.224.254.101/32
set firewall group network-group user1 network 10.225.254.101/32
set firewall group network-group user1 network 10.226.254.101/32
set firewall group network-group user1 network 10.227.254.101/32
set firewall group network-group user1 network 10.10.0.0/20

i use the same network group as the source and destination and i apply it to the in and out on all 4 vtun interfaces.

it works to get access to the servers, but the problem arrises when a VPN user wants to access a remote site that’s connected to a different VPN server…

i think i would need to put all 16 possible IP’s into the list on every router for every user… so that’s like a minimum of 17 firewall “things” to go thru for each and every of the (possibly) thousands of users…

am I missing something simple like being able to just make a network-group that says 10.x.254.101 or 10.[224-239].254.101 ?

any other suggestions on how to cut down the number of firewall stuff to define? is there a way that maybe I can get the vpn servers to all use the same subnets without NAT?