I wonder if it is driver-related: as I understand it, VyOS 1.4 and 1.5 both use the in-tree Linux Kernel ixgbe driver by default, which is not the same as the ixgbe driver provided “out-of-tree” by Intel.
My experience with ixgbe and VyOS is limited to the newer X553 interface, where I encountered some link issues with the in-tree ixgbe driver, similar to what is detailed on this thread.
@samip537 there was building 1.4 images incorporating the very latest ixgbe driver from Intel, and making them available here. If you are OK with testing with one of those, that might shed some more light on the issue.