Yes, you can certainly do so, however “how” you do it depends on your current network architecture, the amount of “down-time” risk you want to assume, etc.
I would advise one of the following approaches:
(A) (least risky):
- create a completely new network (i.e. VLAN) for playing with VyOS;
- configure VyOS to route, DHCP and NAT that network;
- the VyOS “uplink” can be placed in an existing network served by your pfSense;
- in this way you have zero impact on your current pfSense;
(B) (some risks):
- create a new VLAN which you configure both on pfSense and VyOS as a “routers” network;
- make sure that VyOS uses pfSense as the default router, and that pfSense uses VyOS as the next hop to the networks delegated to VyOS;
- choose a network to move to VyOS;
- disable that network (most likely a VLAN) from pfSense;
- configure that network on VyOS (only routing and DHCP);
- minimal impact, if you need to revert, just re-enable the interface on pfSense and disable the interface on VyOS;
There might be other approaches, but it depends highly on your network topology.