Can not establish SSH connection from vyos device through VRF interface

[vxlan]

Hi all,

We’ve noticed that there is an unexpected issue when trying to perform an ssh/scp from vyos to other destination when being routed through VRF.

In our scenario we have vyos with eth0 in network_A, with vrf for management. We’ve also have 2 other interfaces for lan/wan. We want to ssh/scp from vyos management to other server in network_B, for that we’ve configured a static route. We can ping the other host, and performing a tcpdump we capture the SYN and SYN ACK packets, but after that, the vyos sends a RESET and is not possible to establish the connection.

Is there a kind of bug for this situation regarding VRF and handshake? It seems that there is no routing issue at all…

here more details regarding configuration:

set interfaces ethernet eth0 vrf 'management'
set protocols static route DESTINATION_IP next-hop VRF_GATEWAY next-hop-vrf 'management'
set protocols vrf management static route 0.0.0.0/0 next-hop VRF_GATEWAY
set service ssh vrf 'management'
set vrf name management table '1'

and the tcpdump performed in vyos eth0:

10:37:54.429608 IP VYOS_VRF_IP.36294 > DESTINATION_IP.22: Flags [S], seq 3067936304, win 64240, options [mss 1460,sackOK,TS val 2474142607 ecr 0,nop,wscale 7], length 0
10:37:54.463050 IP DESTINATION_IP.22 > VYOS_VRF_IP.36294: Flags [S.], seq 1420136728, ack 3067936305, win 28960, options [mss 1460,sackOK,TS val 1310452683 ecr 2474142607,nop,wscale 7], length 0
10:37:54.463076 IP VYOS_VRF_IP.36294 > DESTINATION_IP.22: Flags [R], seq 3067936305, win 0, length 0

Many thanks and regards

[Apachez]

I have my VRF config isolated from each other as in no VRF leaking (your “set protocols static … next-hop VRF…” is a VRF leak setup).

Here are the current config I use in the lab:

set interfaces ethernet eth0 address '192.168.56.2/24'
set interfaces ethernet eth0 description 'MGMT'
set interfaces ethernet eth0 duplex 'auto'
set interfaces ethernet eth0 hw-id '<REMOVED>'
set interfaces ethernet eth0 ip arp-cache-timeout '240'
set interfaces ethernet eth0 offload gro
set interfaces ethernet eth0 offload gso
set interfaces ethernet eth0 offload lro
set interfaces ethernet eth0 offload rfs
set interfaces ethernet eth0 offload rps
set interfaces ethernet eth0 offload sg
set interfaces ethernet eth0 offload tso
set interfaces ethernet eth0 ring-buffer rx '4096'
set interfaces ethernet eth0 ring-buffer tx '4096'
set interfaces ethernet eth0 speed 'auto'
set interfaces ethernet eth0 vrf 'VRF_MGMT'

set interfaces ethernet eth1 address '192.168.1.2/24'
set interfaces ethernet eth1 description 'WAN'
set interfaces ethernet eth1 duplex 'auto'
set interfaces ethernet eth1 hw-id '<REMOVED>'
set interfaces ethernet eth1 ip arp-cache-timeout '240'
set interfaces ethernet eth1 offload gro
set interfaces ethernet eth1 offload gso
set interfaces ethernet eth1 offload lro
set interfaces ethernet eth1 offload rfs
set interfaces ethernet eth1 offload rps
set interfaces ethernet eth1 offload sg
set interfaces ethernet eth1 offload tso
set interfaces ethernet eth1 ring-buffer rx '4096'
set interfaces ethernet eth1 ring-buffer tx '4096'
set interfaces ethernet eth1 speed 'auto'
set interfaces ethernet eth1 vrf 'VRF_TEST'

set service ssh access-control allow user 'vyos'
set service ssh ciphers 'aes256-gcm@openssh.com'
set service ssh dynamic-protection allow-from '192.168.0.0/16'
set service ssh dynamic-protection block-time '60'
set service ssh dynamic-protection detect-time '3600'
set service ssh dynamic-protection threshold '10'
set service ssh listen-address '192.168.56.2'
set service ssh mac 'hmac-sha2-512'
set service ssh rekey data '1024'
set service ssh rekey time '60'
set service ssh vrf 'VRF_MGMT'

set system option http-client source-interface 'eth1'
set system option ssh-client source-interface 'eth1'

set vrf name VRF_MGMT description 'Management'
set vrf name VRF_MGMT table '100'
set vrf name VRF_MGMT vni '10000100'
set vrf name VRF_TEST description 'Test'
set vrf name VRF_TEST protocols static route 0.0.0.0/0 next-hop 192.168.1.254 distance '1'
set vrf name VRF_TEST table '101'
set vrf name VRF_TEST vni '10000101'

Then to select which VRF to use for outgoing SSH connections from the VyOS box I use this script as example (placed in /config/custom/config_backup.sh):

#!/bin/sh

# Script debugging
set -x

# Set variables
NOW=$(date +"%y%m%d_%H%M")
SRCFILE=/config/config.boot
DSTFILE=/home/username/vyos/config_${NOW}.boot
USER=username
SERVER=192.168.1.100
VRF=VRF_TEST

# Perform stuff
ip vrf exec ${VRF} scp ${SRCFILE} ${USER}@${SERVER}:${DSTFILE}

The magic is the line at the bottom (example if you from CLI want to SSH to a different host on the VRF_MGMT):

ip vrf exec VRF_MGMT ssh user@host

[vxlan]

Hi @Apachez ,
Many thanks for your smart solution, it definitively works!
Kind regards!