Oh, I can see in latest that negated groups can be used in firewall and policy, even though it’s not listed in cli! You need to negate using character !
set policy route VPN_v4_BYPASS rule 110 set table '100'
set policy route VPN_v4_BYPASS rule 110 source group address-group '!NO_VPN_v4_BYPASS'