Thanks to your replay @16again @n.fort @fernando 
My VyOS version is
vyos@vyos:~$ show version
Version: VyOS 1.4-rolling-202112211328
Release train: sagitta
Built by: [email protected]
Built on: Tue 21 Dec 2021 13:28 UTC
Build UUID: 22a9f692-6e20-4886-8bba-c2f51a59d217
Build commit ID: f84a69729ad517
Architecture: x86_64
Boot via: installed image
System type: KVM guest
My configuration for all routers
VyOs 1 ( The left router)
set interfaces ethernet eth0 address '172.16.1.1/24'
set interfaces ethernet eth0 hw-id '50:00:00:12:00:00'
set interfaces ethernet eth1 address '172.16.2.1/24'
set interfaces ethernet eth1 hw-id '50:00:00:12:00:01'
set interfaces ethernet eth2 address '192.168.1.2/24'
set interfaces ethernet eth2 hw-id '50:00:00:12:00:02'
set protocols ospf area 0 network '172.16.1.0/24'
set protocols ospf area 0 network '172.16.2.0/24'
set protocols ospf redistribute connected
VyOS 2 ( The right router)
set interfaces ethernet eth0 hw-id '50:00:00:11:00:00'
set interfaces ethernet eth1 address '172.16.4.1/24'
set interfaces ethernet eth1 hw-id '50:00:00:11:00:01'
set interfaces ethernet eth2 address '10.10.10.2/24'
set interfaces ethernet eth2 hw-id '50:00:00:11:00:02'
set protocols ospf area 0 network '172.16.3.0/24'
set protocols ospf area 0 network '172.16.4.0/24'
set protocols ospf redistribute connected
VyOS 19
set interfaces ethernet eth0 address '172.16.2.2/24'
set interfaces ethernet eth0 hw-id '50:00:00:13:00:00'
set interfaces ethernet eth1 address '172.16.4.2/24'
set interfaces ethernet eth1 hw-id '50:00:00:13:00:01'
set protocols ospf area 0 network '172.16.2.0/24'
set protocols ospf area 0 network '172.16.4.0/24'
VyOS 20
set interfaces ethernet eth0 address '172.16.1.2/24'
set interfaces ethernet eth0 hw-id '50:00:00:14:00:00'
set interfaces ethernet eth1 address '172.16.3.2/24'
set interfaces ethernet eth1 hw-id '50:00:00:14:00:01'
set protocols ospf area 0 network '172.16.1.0/24'
set protocols ospf area 0 network '172.16.3.0/24'
VyOS 1 ( the left router)
vyos@vyos# run show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
O>* 10.10.10.0/24 [110/20] via 172.16.1.2, eth0, weight 1, 19:13:51
* via 172.16.2.2, eth1, weight 1, 19:13:51
O 172.16.1.0/24 [110/1] is directly connected, eth0, weight 1, 19:14:47
C>* 172.16.1.0/24 is directly connected, eth0, 19:14:54
O 172.16.2.0/24 [110/1] is directly connected, eth1, weight 1, 19:14:47
C>* 172.16.2.0/24 is directly connected, eth1, 19:14:55
O>* 172.16.3.0/24 [110/2] via 172.16.1.2, eth0, weight 1, 19:14:01
O>* 172.16.4.0/24 [110/2] via 172.16.2.2, eth1, weight 1, 19:13:57
C>* 192.168.1.0/24 is directly connected, eth2, 19:14:53
VyOS 2 ( the right router)
vyos@vyos# run show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
C>* 10.10.10.0/24 is directly connected, eth2, 19:15:43
O>* 172.16.1.0/24 [110/2] via 172.16.3.2, eth0, weight 1, 19:14:48
O>* 172.16.2.0/24 [110/2] via 172.16.4.2, eth1, weight 1, 19:14:53
O 172.16.3.0/24 [110/1] is directly connected, eth0, weight 1, 19:15:37
C>* 172.16.3.0/24 is directly connected, eth0, 19:15:44
O 172.16.4.0/24 [110/1] is directly connected, eth1, weight 1, 19:15:37
C>* 172.16.4.0/24 is directly connected, eth1, 19:15:44
O>* 192.168.1.0/24 [110/20] via 172.16.3.2, eth0, weight 1, 19:14:47
* via 172.16.4.2, eth1, weight 1, 19:14:47
[edit]
When I ping VPC22 from VPC21, I can see the Ping request go through the down path and the response come back from the top path. However, this does not imply that it can load balance between interfaces. Moreover, when I enable WAN load balancing, it works properly, indicating that both paths contain the ping request and response on their own. is there any miss-configuration?
Thank you so much.