Routing between vlans is not working between hosts connected to vyos

groovenarula · December 10, 2024, 9:49pm

Hi All.

I am trying to ping from my main ‘LAN’ to a VLAN and just can’t seem to get it work.

My interfaces are setup as follows :

set interfaces ethernet eth1 address '192.168.2.1/24'
set interfaces ethernet eth1 description 'LAN'
set interfaces ethernet eth1 hw-id '90:e2:ba:3f:f2:69'
set interfaces ethernet eth1 vif 172 address '172.16.0.1/12'
set interfaces ethernet eth1 vif 172 description 'servers'

I have included both interfaces in my ‘trust’ zone :

set firewall zone trust default-action 'drop'
set firewall zone trust from local firewall name 'local-trust'
set firewall zone trust from untrust firewall name 'untrust-trust'
set firewall zone trust interface 'pod-cniBridge'
set firewall zone trust interface 'eth1'
set firewall zone trust interface 'eth1.172'

Output of show interfaces :

Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address          MAC                VRF        MTU  S/L    Description
-----------  ------------------  -----------------  -------  -----  -----  ---------------------------------------------------------
eth0         192.168.11.11/24    10:c4:ca:a6:16:50  default   1500  u/u    ATT-XSGPON
             xxx.xxx.xxx.xxx/23
eth1         192.168.2.1/24      90:e2:ba:3f:f2:69  default   1500  u/u    LAN
eth1.172     172.16.0.1/12       90:e2:ba:3f:f2:69  default   1500  u/u    servers
lo           127.0.0.1/8         00:00:00:00:00:00  default  65536  u/u
             ::1/128
veth0        -                   56:d7:ee:63:52:76  default   1500  u/u
veth1        -                   6a:81:cb:2a:2f:2a  default   1500  u/u
veth2        -                   92:0f:c3:6a:8e:72  default   1500  u/u

Output of show ip route :

Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

S>* 0.0.0.0/0 [210/0] via xxx.xxx.xxx.xxx, eth0, weight 1, 17:53:51
C>* xxx.xxx.xxx.0/23 is directly connected, eth0, 17:53:55
C>* 172.16.0.0/12 is directly connected, eth1.172, 17:54:00
C>* 192.168.2.0/24 is directly connected, eth1, 17:54:00

I can ping a host on 172 vlan when logged into the vyos host:

vyos@vyos:~$ ping 172.16.5.1
PING 172.16.5.1 (172.16.5.1) 56(84) bytes of data.
64 bytes from 172.16.5.1: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 172.16.5.1: icmp_seq=2 ttl=64 time=0.182 ms
64 bytes from 172.16.5.1: icmp_seq=3 ttl=64 time=0.191 ms
64 bytes from 172.16.5.1: icmp_seq=4 ttl=64 time=0.239 ms
64 bytes from 172.16.5.1: icmp_seq=5 ttl=64 time=0.227 ms
64 bytes from 172.16.5.1: icmp_seq=6 ttl=64 time=0.184 ms
^C
--- 172.16.5.1 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5146ms
rtt min/avg/max/mdev = 0.158/0.196/0.239/0.027 ms

But I am unable to ping this host from any other host on 192.168.2.0/24 network.

All hosts on the 192.168.2.0/24 network can route and ping hosts on the Internet :

vyos@vyos:~$ ping one.one.one.one
PING one.one.one.one (1.1.1.1) 56(84) bytes of data.
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=1 ttl=58 time=13.1 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=2 ttl=58 time=13.1 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=3 ttl=58 time=13.3 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=58 time=13.2 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=5 ttl=58 time=13.2 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=58 time=15.5 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=7 ttl=58 time=13.1 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=8 ttl=58 time=14.8 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=9 ttl=58 time=13.6 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=10 ttl=58 time=14.9 ms
64 bytes from one.one.one.one (1.1.1.1): icmp_seq=11 ttl=58 time=12.9 ms
^C
--- one.one.one.one ping statistics ---
11 packets transmitted, 11 received, 0% packet loss, time 10016ms
rtt min/avg/max/mdev = 12.936/13.702/15.539/0.872 ms

For sure I am missing something simple - but can’t seem to pin-point what I missing. I had this working on another instance of vyos that crashed (flash died).

I am the following version of vyos:

vyos@vyos:~$ show version
Version:          VyOS 1.4-rolling-202403042245
Release train:    sagitta

Built by:         [email protected]
Built on:         Mon 04 Mar 2024 22:45 UTC
Build UUID:       b2f926d2-e65f-4973-b0cb-070616cea02f
Build commit ID:  d42f7e8a9e79da

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:
Hardware UUID:    76aaa1be-4633-4586-a155-e02926bf96b2

Copyright:        VyOS maintainers and contributors

It’s running as a proxmox VM with NIC’s passed through to the VM.

Would really appreciate any help with how to troubleshoot this issue.

Thanks in advance.

tjh · December 10, 2024, 10:06pm

Change your default action to allow.
Does that fix it? Then you need to check your fw rules closer.

groovenarula · December 10, 2024, 10:28pm

Thanks for the quick reply.

Looks like my version of vyos does not allow ‘accept’ / ‘allow’ as a default action in the zone :

vyos@vyos# set firewall zone trust default-action
Possible completions:
   drop                 Drop silently (default)
   reject               Drop and notify source



[edit]

I removed the default action rule statement completely. But that did not resolve the issue.
The complete firewall section is now as follows:

vyos@vyos:~$ show configuration commands | grep firewall
set firewall ipv4 name local-trust default-action 'accept'
set firewall ipv4 name local-untrust default-action 'accept'
set firewall ipv4 name trust-local default-action 'accept'
set firewall ipv4 name trust-untrust default-action 'accept'
set firewall ipv4 name untrust-local default-action 'drop'
set firewall ipv4 name untrust-local rule 10 action 'accept'
set firewall ipv4 name untrust-local rule 10 description 'Allow EST/Related Traffice'
set firewall ipv4 name untrust-local rule 10 state 'established'
set firewall ipv4 name untrust-local rule 10 state 'related'
set firewall ipv4 name untrust-local rule 20 action 'accept'
set firewall ipv4 name untrust-local rule 20 protocol 'icmp'
set firewall ipv4 name untrust-local rule 20 state 'new'
set firewall ipv4 name untrust-local rule 100 action 'accept'
set firewall ipv4 name untrust-local rule 100 destination port '51820'
set firewall ipv4 name untrust-local rule 100 protocol 'udp'
set firewall ipv4 name untrust-local rule 100 state 'new'
set firewall ipv4 name untrust-trust default-action 'drop'
set firewall ipv4 name untrust-trust rule 10 action 'accept'
set firewall ipv4 name untrust-trust rule 10 description 'Allow EST/Related Traffice'
set firewall ipv4 name untrust-trust rule 10 state 'established'
set firewall ipv4 name untrust-trust rule 10 state 'related'
set firewall ipv4 name untrust-trust rule 20 action 'accept'
set firewall ipv4 name untrust-trust rule 20 protocol 'icmp'
set firewall ipv4 name untrust-trust rule 20 state 'new'
set firewall zone local default-action 'drop'
set firewall zone local from trust firewall name 'trust-local'
set firewall zone local from untrust firewall name 'untrust-local'
set firewall zone local local-zone
set firewall zone trust from local firewall name 'local-trust'
set firewall zone trust from untrust firewall name 'untrust-trust'
set firewall zone trust interface 'eth1'
set firewall zone trust interface 'eth1.172'
set firewall zone untrust default-action 'drop'
set firewall zone untrust from local firewall name 'local-untrust'
set firewall zone untrust from trust firewall name 'trust-untrust'
set firewall zone untrust interface 'eth0'

Thanks again.

son · December 19, 2024, 7:57am

try to add a rule to explicitly allow all traffic from the 192.168.2.0/24 network to the 172.16.0.0/12 network under local-trust ruleset.

16again · December 22, 2024, 5:41pm

Seems like traffic between members of same zone is broken. As reviewing created nftables rules could confirm.
Try allowing using rule like
set firewall zone trust from trust firewall name ‘trust-trust’
And ruleset trust-trust is a allow all