Ipsec issue . tunnel is up but no traffic/ no routing

Bugs
, , ,
1

Hi,
Not sure if this is a bug or not, so if not, please move to the correct category.

vyos@vyos:~$ show version 

Version:          VyOS 1.3-rolling-202006241940
Release Train:    equuleus

Built by:         [email protected]
Built on:         Wed 24 Jun 2020 19:40 UTC
Build UUID:       35acaac7-e83c-430d-915f-8e63670c8178
Build Commit ID:  f2f6332ca1acd9

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  OpenStack Foundation
Hardware model:   OpenStack Nova
Hardware S/N:     Unknown
Hardware UUID:    Unknown

Copyright:        VyOS maintainers and contributors

Interfaces

vyos@vyos:~$ show interfaces 
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth1             10.10.100.1/24                  u/u  
eth2             A.A.A.A/24                  u/u  
lo               127.0.0.1/8                       u/u  
                 ::1/128

Relevant Config

set vpn ipsec esp-group remote-esp compression 'disable'
set vpn ipsec esp-group remote-esp lifetime '3600'
set vpn ipsec esp-group remote-esp mode 'tunnel'
set vpn ipsec esp-group remote-esp pfs 'disable'
set vpn ipsec esp-group remote-esp proposal 1 encryption 'aes256'
set vpn ipsec esp-group remote-esp proposal 1 hash 'sha1'

set vpn ipsec ike-group remote-ike ikev2-reauth 'no'
set vpn ipsec ike-group remote-ike key-exchange 'ikev1'
set vpn ipsec ike-group remote-ike lifetime '28800'
set vpn ipsec ike-group remote-ike proposal 1 dh-group '2'
set vpn ipsec ike-group remote-ike proposal 1 encryption '3des'
set vpn ipsec ike-group remote-ike proposal 1 hash 'sha1'

set vpn ipsec ipsec-interfaces interface 'eth2'
set vpn ipsec nat-traversal 'enable'

set vpn ipsec site-to-site peer B.B.B.B authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer B.B.B.B authentication pre-shared-secret 'secret'
set vpn ipsec site-to-site peer B.B.B.B default-esp-group 'remote-esp'
set vpn ipsec site-to-site peer B.B.B.B ike-group 'remote-ike'
set vpn ipsec site-to-site peer B.B.B.B local-address 'A.A.A.A'

set vpn ipsec site-to-site peer B.B.B.B tunnel 1 local prefix '10.10.100.0/24'
set vpn ipsec site-to-site peer B.B.B.B tunnel 1 remote prefix '10.10.0.0/24'

Tunnel is up

vyos@vyos:~$ show vpn ipsec sa 
Connection                State    Uptime    Bytes In/Out    Packets In/Out    Remote address    Remote ID    Proposal
------------------------  -------  --------  --------------  ----------------  ----------------  -----------  ------------------------
peer-B.B.B.B-tunnel-1  up       10m15s    0B/0B           0/0               B.B.B.B        N/A          AES_CBC_256/HMAC_SHA1_96

Looks normal

vyos@vyos:~$ show vpn debug 
Status of IKE charon daemon (strongSwan 5.7.2, Linux 4.19.128-amd64-vyos, x86_64):
  uptime: 11 minutes, since Jun 30 15:14:40 2020
  malloc: sbrk 2945024, mmap 0, used 1119632, free 1825392
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 2
  loaded plugins: charon test-vectors ldap pkcs11 tpm aesni aes rc2 sha2 sha1 md5 mgf1 rdrand random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default connmark stroke vici updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp lookip error-notify certexpire led addrblock counters
Listening IP addresses:
  A.A.A.A
Connections:
peer-B.B.B.B-tunnel-1:  A.A.A.A...B.B.B.B  IKEv1
peer-B.B.B.B-tunnel-1:   local:  [A.A.A.A] uses pre-shared key authentication
peer-B.B.B.B-tunnel-1:   remote: [B.B.B.B] uses pre-shared key authentication
peer-B.B.B.B-tunnel-1:   child:  10.10.100.0/24 === 10.10.0.0/24 TUNNEL

Security Associations (1 up, 0 connecting):

peer-B.B.B.B-tunnel-1[1]: ESTABLISHED 11 minutes ago, A.A.A.A[A.A.A.A]...B.B.B.B[B.B.B.B]
peer-B.B.B.B-tunnel-1[1]: IKEv1 SPIs: 085d93c9bdbc5ad8_i* 63bd0b0a10a5488e_r, pre-shared key reauthentication in 7 hours
peer-B.B.B.B-tunnel-1[1]: IKE proposal: 3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024
peer-B.B.B.B-tunnel-1{1}:  REKEYED, TUNNEL, reqid 1, expires in 48 minutes
peer-B.B.B.B-tunnel-1{1}:   10.10.100.0/24 === 10.10.0.0/24
peer-B.B.B.B-tunnel-1{2}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: cf1d245e_i c4a0545c_o
peer-B.B.B.B-tunnel-1{2}:  AES_CBC_256/HMAC_SHA1_96, 0 bytes_i, 0 bytes_o, rekeying in 35 minutes
peer-B.B.B.B-tunnel-1{2}:   10.10.100.0/24 === 10.10.0.0/24

Policy

vyos@vyos:~$ show vpn ipsec policy 
src 10.10.100.0/24 dst 10.10.0.0/24 
        dir out priority 375423 ptype main 
        tmpl src A.A.A.A dst B.B.B.B
                proto esp spi 0xc4a0545c reqid 1 mode tunnel
src 10.10.0.0/24 dst 10.10.100.0/24 
        dir fwd priority 375423 ptype main 
        tmpl src B.B.B.B dst A.A.A.A
                proto esp reqid 1 mode tunnel
src 10.10.0.0/24 dst 10.10.100.0/24 
        dir in priority 375423 ptype main 
        tmpl src B.B.B.B dst A.A.A.A
                proto esp reqid 1 mode tunnel
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket out priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket in priority 0 ptype main 
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket out priority 0 ptype main 
src ::/0 dst ::/0 
        socket in priority 0 ptype main 
src ::/0 dst ::/0 
        socket out priority 0 ptype main 
src ::/0 dst ::/0 
        socket in priority 0 ptype main 
src ::/0 dst ::/0 
        socket out priority 0 ptype main

PROBLEM.
10.10.100.50/24 => VYOS [10.10.100.1] <internet> [B.B.B.B] -- 10.10.0.1/24

The default gateway for 10.10.100.50 is 10.10.100.1, which is the vyos.
The tunnel is up, but both sides cannot ping each other.

If I do trace-route 10.10.0.1 from 10.10.100.50, it tries to go out from the default router of the vyos … and does not hit 10.10.0.1 or B.B.B.B
The IP tries to go via the default gateway in the vyos and not via the tunnel.
Bytes in/out over the tunnel is zero.

There are no firewalls anywhere.

The other end is a cisco I do not have access to.

2

Hi @admin0, can you check table 220

show ip route table 220

Did you add IP address on eth0 after configuring ipsec? If yes, try restart vpn

3

Hi Dmitry,

Thank you for the quick reply.

I setup the IP addresses first, before working on the ipsec.
Here is my table

vyos@vyos:~$ show ip route table  220
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF default table 220:
K>* 10.10.0.0/24 [0/0] via A.B.C.D{default gateway of vyos}, eth2, src 10.10.100.1, 00:33:56
4

I did the restart vpn.

tunnel is up.
The route table 220 is the same …
10.10.0.0/24 via {default gw of vyos }

5

Really interesting situation, I will try to reproduce soon. Check please all routes, maybe you have a more specific route?

6

The sole purpose of this vyos is to enable that ipsec connection and allow the local 10.10.100.0/24 talk to 10.10.0.0/24

vyos@vyos:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route

S>* 0.0.0.0/0 [1/0] via A.A.A.1, eth2, 5d02h26m
C>* A.A.A.0/24 is directly connected, eth2, 5d02h26m
C>* 10.10.100.0/24 is directly connected, eth1, 5d02h26m

I do not have any static routes. Only line is:

set protocols static route 0.0.0.0/0 next-hop A.A.A.1

7

I do have this

set nat source rule 11 outbound-interface 'eth2'
set nat source rule 11 source address '10.10.100.0/24'
set nat source rule 11 translation address 'masquerade'

But this is for packets leaving to the internet. From the ipsec settings, 10.10.0.0/24 should be routed to the tunnel right ?

8

Hello @admin0, yes, I can confirm that this related to source NAT rules. Try to add exclude to your NAT rules

set nat source rule 11 destination address '!10.10.0.0/24'