According to the template at least in 1.4-rolling there is the option to through VyOS config to disable_external_cache:
set service conntrack-sync disable-external-cache
It also seems to exist for VyOS 1.3: