Hello @bernd.dausch .
Can you provide firewall logs on side UTM in this time? Maybe UTM drop this packets.
Check that the port 500 is open on the UTM side.
From VyOS execute:
vyos# sudo nmap -sU -p 500 x.x.x.x
vyos# sudo nmap -sU -p 4500 x.x.x.x
Where x.x.x.x - UTM host.