Simple source routing not so simple? (How to migrate this simple RouterOS example to VyOS)

General questions
1

I am struggling for hours with what I thought is an easy thing: Source routing. I am new ro VyOS and used Mikrotik RouterOS before. Here is what I had there:

/ip route rule add dst-address=0.0.0.0/0 src-address=192.0.2.0/24 table=main
/ip route rule add dst-address=0.0.0.0/0 src-address=192.0.2.0/24 table=default_pub
/ip route rule add dst-address=0.0.0.0/0 table=main
/ip route rule add dst-address=0.0.0.0/0 table=default_lan

where 192.0.2.0/24 is my public /24 and I want to achieve that everything with a source address from 192.0.2.0/24 is not sent over the normal default gateway but a separate tunnel.
For this reason, I removed the default gateway from the main routing table (“main”) and moved it to “default_lan”. Then I created a table “default_pub” that only includes a default route via the separate tunnel (which routes the 192.0.2.0/24).

What above commands do:

  1. For any source address 192.0.2.0/24, we first try the main routing table. If there are no matches, we try table default_pub. (It must match because this table has a default route)
  2. For any other address, we first try the main routing table. If there are no matches, we try default_lan (which must match because it includes the default gateway)

I tried the same thing in VyOS. To simplify things, I just try “local-route”. Here, table 170 corresponds to default_lan in the example above.

 policy {
     local-route {
         rule 101 {
             destination 0.0.0.0/0
             set {
                 table main
             }
         }
         rule 102 {
             destination 0.0.0.0/0
             set {
                 table 170
             }
         }
     }
 }

In my opinion, all locally generated packets should now reach the internet. But they don’t. It seems only rule 101 is processed but table 170 is never consulted.

2

As I remember there are no “try”.
If a packet match rule 101 it won’t use the next rule.
You can extend policy-local route match source/destination or marking.
I think mikrotik works the same way

“sudo ip rule show” shows the current rules

3

Hmm, interesting, but this definitely works on RouterOS.

How can I implement this then? Let me repeat and clarify what I am looking for. This is my current routing table:

vyos@SunGate1:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S>* 0.0.0.0/0 [210/0] via 181.183.144.1, eth0.2, weight 1, 00:00:10
C>* 10.227.1.0/24 is directly connected, eth0.5, 00:58:04
C>* 10.227.2.0/24 is directly connected, eth0.4, 00:58:05
C>* 10.227.79.0/24 is directly connected, eth0.3, 00:58:05
C>* 10.227.80.0/24 is directly connected, eth0.10, 00:58:03
O   192.0.2.226/31 [110/100] is directly connected, wg0, weight 1, 00:57:51
C>* 192.0.2.226/31 is directly connected, wg0, 00:57:55
C>* 192.168.222.0/24 is directly connected, eth0.11, 00:58:03
C>* 192.168.223.0/24 is directly connected, eth0.12, 00:58:02
C>* 181.183.144.0/21 is directly connected, eth0.2, 00:00:11
vyos@SunGate1:~$

I want that if the source address of any packet (local or not local) is 192.0.2.0/24, the default route should not be used. Instead it should be routed via nexthop 192.0.2.226.

As I said, in RouterOS I could solve this by removing the default gateway from this table and put it into a separate table. The rules would then consult the main table and then the default gw tables (based on source address).

Maybe there is a better/different way to do this in VyOS…

4

You don’t set specified source

vyos@r11# set policy local-route rule 101 
Possible completions:
+  destination  Destination address or prefix
   fwmark       Match fwmark value
   inbound-interface
                Inbound Interface
 > set          Packet modifications
+  source       Source address or prefix

I.e

 set policy local-route rule 101 source 192.0.2.0/24
 set policy local-route rule 101 destination x.x.x.x/x
 set policy local-route rule 101 set table xxx
5

You mean I should directly jump into table 170?
But this is exactly my problem: This table only contains the default gateway, not all the other routes (interface, static, OSPF).

Any packet should still first try all the entries from the main routing table. For example, a packet from 192.168.0.2.15 to 10.227.2.10 should of course use the interface route that’s already in the main table.

Now I could manually copy each and every entry from the main routing table into table 170 but I hope it’s obvious that this is not an option, especially when the routing table becomes large and complex. It’s only the default route which should be changed based on source address.

6 
vyos@r11# run show conf com | match rule
set policy local-route rule 101 set table '101'
set policy local-route rule 101 source '192.0.2.0/24'
[edit]
vyos@r11# 
[edit]
vyos@r11# 
[edit]
vyos@r11# sudo ip rule show
0:	from all lookup local
101:	from 192.0.2.0/24 lookup 101
32766:	from all lookup main
32767:	from all lookup default
[edit]
vyos@r11#

The rule zero will search from all local addresses (local table, there are directly connected to this host)
After this, it will search next rule 101, and if it matches, it will send traffic to table 101, otherwise, use rule 32766 and check the “main” table

7

Two questions.
Question 1: how did you get rule 0? For me it looks like this (I believe I used the same VyOS commands as you):


vyos@SunGate1# sudo ip rule show 
101:from 192.0.2.0/24 lookup 101
1000:   from all lookup [l3mdev-table]
2000:   from all lookup [l3mdev-table] unreachable
32765:  from all lookup local
32766:  from all lookup main
32767:  from all lookup default
[edit]
vyos@SunGate1#

Question 2: I believe this does still not solve my fundamental question/problem: Your rule for the main table (#32766) is AFTER rule #101 which will always match. But the main table includes OSPF routes, static routes etc. So these routes are never consulted if the source address 192.0.2/24 but they should. Or am I wrong?

8

If you know what traffic should still use main routing table, then you can create multiple entries in policy local route:

  • First n rules: match all known traffic that should still use routing table ( for example source 192.168.0.215 to 10.227.2.10)
  • Last policy local route: match all other traffic and set table x
9

Rule 0 by default
But I didn’t use vrf in my example

10

rule 0 normally is present, but at exp, it ends up as 32765. Don’t ask why.
How about not having a default route in main at all?
Then start lowest policy using table main,
then 2 rules , each pointing to different table, having its own default gateway

11

But then I would replicate whatever is in the routing table with rules. Everything should always use the main routing table (except the default route entry)

Is it possible to move the rule for local and main to a lower value?

Then I could add my specific default routes afterwards.

12

Which version are you using? I am using 1.3. Maybe that’s the reason?

This is exactly what I am having right now (see the original posting).
But it seems table 170 (containing the default route) is not consulted …

How would you configure this? Maybe I am doing something wrong…

13

But in your 2nd post, main route has a default route. So ip rule 102 (and table 170) will never come into play

14

That is right, very sorry for the confusion. My posting is not consistent. There should be NO default route in the 2nd posting.
The reason why it’s there is because I re-stated the original problem from the beginning … assuming I would start with a normal table that includes the default route.