1.2.6-S1: "show ip route" doesn't match underlying Debian "ip route" - Traffic misbehaving

Hey folks,

I’ve had VyOS running in ESXi 6.7 for over a year now, and I’ve just run into a problem recently where VyOS is sending traffic out the wrong interface. I’ve found that there is a mismatch between VyOS’s route table and the output of the OS’s ip route command.

Apologies for the images - I can’t SSH in so I’m stuck using the ESXi web console, which doesn’t support copy/paste :slight_smile:

Here are the details:

  1. Version output (using the VyOS official ISO for 1.2.6-S1): https://imgur.com/E5NWarN
  2. Interface config (eth0 is DHCP-connected WAN, eth1 is a LAN /30, BGP’d to a Cisco 3750x, though we’re not connected to even get BGP up): https://imgur.com/y6LDV4k
  3. Route output (I have some VPNs with VTIs that are also misbehaving and going out eth0 instead of getting into the VTI, but I suspect its the same issue): https://imgur.com/3DM8BeA

Issue Overview:

  • Traffic destined for (which is the other end of the eth1 /30) is going out eth0 instead. I can reproduce this with ping, but since I can’t have two windows open at once at the moment due to the router being disconnected, here’s a repro with BGP:
  • We can see, however - that the VyOS route table shows the proper route, but ip route shows eth0: https://imgur.com/60N2DVo

Thoughts on why this is happening? I’ve rebooted a few times, and over time, this ends up happening. My other VyOS router (running in a primary/failover model via BGP) is just fine with about the exact same config. Anything I can check or pull to debug this?


As I can see in image Imgur: The magic of the Internet , the route assigned to dst address is table 220.
No extra config related to routing table was made in router?

Nothing outstanding. Few static interface routes (none related to

Also to note, route table 220, which I believe is supposed to have zero prefixes, only has the one kernel route: https://imgur.com/oTpJNxu

I believe this is overriding the table 0 routes and causing my issue.

On its sister router (same config, different /30 eth1 LAN prefix), route table 220 has no routes and works great. Is there something that’d cause that route to be added to table 220?

No differences in config between the two routers, aside from some IPs and a BGP thing or two (as path route map added, etc). Nothing under protocols static, etc.

Here all all my static routes, they only exist for VPN VTIs:

set protocols static interface-route next-hop-interface vti2
set protocols static interface-route next-hop-interface vti3
set protocols static interface-route next-hop-interface vti4
set protocols static interface-route next-hop-interface vti0
set protocols static interface-route next-hop-interface vti1
set protocols static interface-route next-hop-interface vti11
set protocols static interface-route next-hop-interface vti10

IPSec added routes to table 220.
Check settings regarding vpn ipsec, local prefix with any peer config.

Ok there was no reference in my config to when grepped against show conf commands. However when I removed the VPN config entirely (just nuked it via delete vpn ipsec), everything came back up.

I then re-added the VPN config, and everything stayed up. Table 220 is now empty, and communication works as expected. Will report back if anything changes.


Update: After 8ish hours (when VPN re-keyed), I now have route table 220 back, with a default route overriding the main route table, sending all traffic out eth0 instead of eth1.

Going to start fresh on 1.2.7 with a fresh config to see if starting clean sorts things out.

Do you use routing-based VPN i.e via vti interfaces only?

Yes, I use VTIs for all IPSec VPN tunnels. I vastly prefer it for my scenario (have since the EdgeOS 1.x days before I moved to VyOS). Could this be causing the issue?

Right as I look at this today, it breaks again :slight_smile:

Now on 1.2.7, same exact issue - table 220 has appeared and overridden the routes again.

Thoughts on where to go now to debug this?

Add this command:
set vpn ipsec options disable-route-autoinstall
Commit/save and reboot the router.

Will test this and report back in a few days if this resolves it. My other router (with the same VTI configs) doesn’t seem to have this issue, but reading the blog post from 2018 regarding this seems to indicate that it should be having this issue :slight_smile:

Thanks @Viacheslav for your help thus far, will let you know how it goes.

@SlothCroissant Did it help you?

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.