PBR question -- traffic coming in an interface must leave from the same interface .. how?

Hi,

my gateway will only allow traffic when the traffic comes from the assigned mac address and ip. Thus I have 3 interfaces in the same sub-net >> eth2, eth3 and eth6.

What I want is, any traffic that comes in to eth2, must go out from eth2 , and same for eth3 and eth6 … They all have the same gateway.

vyos@vyos:~$ show interfaces 
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             192.168.122.11/24                 u/u  
eth2             x.y.z.61/26                  u/u  
eth3             x.y.z.60/26                  u/u  
eth4             -                                 u/u  
eth5             -                                 u/u  
eth6             x.y.z.62/26                  u/u  
eth7             -                                 u/u  
eth8             -                                 u/u  
eth9             -                                 u/u  
eth10            -                                 u/u  
lo               127.0.0.1/8                       u/u  
                 ::1/128


## this is the default gateway via eth0
set protocols static route 0.0.0.0/0 next-hop 192.168.122.1


##
set policy route route-60 rule 60 set table '60'
## using /32 or /26 made no difference 
set policy route route-60 rule 60 source address 'x.y.z.60/32'

set protocols static table 60 route 0.0.0.0/0 next-hop x.y.z.1 next-hop-interface 'eth3'

#
set interfaces ethernet eth3 policy route 'route60'

with the above code in place, if I ping .60 from my laptop, I do not get a reply back. I see that the vyos receives the ping in eth3, but the outgoing is done via eth0 ( so the policy is not working)
When the above works, I can replicate the same to eth2 and eth6.

The goal is any traffic that comes to .60/.61/.62 address … or any traffic from inside the vyos with this IPs are source IP must go via their respective interface to .1 gateway

Please let me know how to make this work.

Hi @admin0, I think PBR works for non-local traffic. https://phabricator.vyos.net/T439

Hi. Thank you for the reply.
What other options do I have ?

I am looking for an equivalent way in vyos that is possible in linux

Add

252 T1
251 T2
to /etc/iproute2/rt_tables.

Next, set up the routing rules to route incoming and outgoing packets via these tables:

ip route add 10.105.16.0/24 dev eth0 src 10.105.16.100 table T1
ip route add default via 10.105.16.2 dev eth0 src 10.105.16.100 table T1
ip rule add from 10.105.16.100 table T1

ip route add 10.105.16.0/24 dev eth1 src 10.105.16.101 table T2
ip route add default via 10.105.16.2 dev eth1 src 10.105.16.101 table T2
ip rule add from 10.105.16.101 table T2

What the above codes does in linux is, that if traffic comes from eth0, it leaves from eth0 and same for eth1 , and more interfaces, where all of them can be in the same subnet. That is exactly what I am looking for.

Ok, rules from your first post would work for clients traffic, but not for IP address on the router. Local PBR for some reason does not implement.

@admin0 Can you provide a network diagram?

I have a server from a datacenter (where vyos is installed )
The datacenter only accepts IPs from known mac addresses.

I do not have a network diagram for this. But is it required ?

if I ping x.y.z.61 which is in eth2, i want the return traffic from .61 to go back via eth2 … ( and not follow the default gateway of the system)

same for .60 in eth2

@Dmitry said it works for non-local traffic … Is a ping reply ( where ping generate from outside of vyos a non-local traffic ) – I am a bit confused here regarding local/non-local traffic.

network
Can we use this diagram as an example?
Or you have many default gateways?

This one

how about source based routing ?
if source = 10.11.12.60, gateway = 10.11.12.1 via dev eth2
if source = 10.11.12.61 , gateway = 10.11.12.1 via dev eth3
if source = 10.11.12.62 , gateway = 10.11.12.1 via dev eth6

@admin0 Try next commands:

set protocols static table 60 interface-route 0.0.0.0/0 next-hop-interface eth2
set protocols static table 61 interface-route 0.0.0.0/0 next-hop-interface eth3
set protocols static table 62 interface-route 0.0.0.0/0 next-hop-interface eth6

and after commit:

sudo ip rule add from 10.11.12.60 lookup 60 
sudo ip rule add from 10.11.12.61 lookup 61
sudo ip rule add from 10.11.12.62 lookup 62

This did not worked …

How does it know where to route the traffic ? I do not see 10.11.12.1 mentioned anywhere which is the gateway …

Can you add dummy interface on 10.11.12.1 an example with ip address 5.5.5.5/32
and ping ?

ping 10.11.12.60 interface 5.5.5.5
ping 10.11.12.61 interface 5.5.5.5
ping 10.11.12.62 interface 5.5.5.5

And what version of the VyOS do you use?