On further investigation it looks like when you enable the forward filter, an implicit accept is added to the firewall. I can’t add a drop rule to the forwarding chain after the offload, because things will brake. The rules on the zone-based firewall will not get hit anymore, because of the implicit accept.
dec 4 16:19:33 vyos kernel: [ 3162.844117] [ipv4-FWD-filter-default-A]IN=wg01 OUT=eth1 MAC= SRC=10.10.0.2 DST=192.168.50.50 LEN=60 TOS=0x00 PREC=0x00 TTL=127 ID=52964 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=174