Routing between VLANs

Hi All.

I am trying to SSH from one VLAN to another, and have run up against my limit of network understanding.

My network consists of a router-on-a-stick and a VLAN-aware switch, and I am using a zone-based firewall. The following image shows the firewall rules in place between what I consider to be the relevant zones.

What I wish to do is SSH from an administration PC (on VLAN30) to a PC on robot1 (on VLAN20). I am able to ping the router interface of VLAN20 (192.168.20.1) from VLAN30, but am unable to ping any of the PCs on VLAN20. Having read a few posts on inter-vlan routing, it seems that a bridge network may be required, but I am not sure.

Can someone please point me in the right direction? Happy to post the relevant bits of my router configuration file if this helps.

This always helps. You don’t need a bridge if you’re trying to route packets :slight_smile:

This should be super straightforward, but things to check

a) Do all your clients have the default GW set properly?
b) Do you see ARP’s for your clients from the router on the right vlan interfaces?

Tim

Hi Tim.

Thanks for your reply. The live configuration file runs to over 2000 lines - here is what I hope are the most important bits (apologies if it is still a bit long):

firewall {
    all-ping enable
    broadcast-ping disable
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians enable
    receive-redirects disable
    send-redirects disable
    source-validation strict
    syn-cookies enable
    group {
        port-group hypertext-ports {
            description "HTTP/HTTPS ports"
            port 80
            port 443
        }
    }
    ...
    firewall rules here
    ...
}
interfaces {
    ethernet eth0 {
        disable
    }
    ethernet eth1 {
        duplex auto
        speed auto
        vif 10 {
            address 192.168.10.1/24
            description "Infrastructure"
        }
        vif 20 {
            address 192.168.20.1/24
            description "Robot 1"
        }
        vif 30 {
            address 192.168.30.1/24
            description "Administration"
        }
    }
    ethernet eth2 {
        address dhcp
        description "WAN"
        duplex auto
        speed auto
    }
    ethernet eth3 {
        disable
    }
    loopback lo {
    }
}
service {
    dhcp-server {
        disabled false
        hostfile-update enable
        static-arp disable
        use-dnsmasq disable
        shared-network-name vlan20-robot-1 {
            authoritative enable
            subnet 192.168.20.0/24 {
                default-router 192.168.20.1
                dns-server 192.168.20.1
                domain-name robot-1.lab
                lease 86400
                start 192.168.20.2 {
                    stop 192.168.20.254
                }
                static-mapping cb1 {
                    ip-address 192.168.20.2
                    mac-address xx:xx:xx:xx:xx:xx
                }
            }
        }
        shared-network-name vlan30-administration {
            authoritative enable
            subnet 192.168.30.0/24 {
                default-router 192.168.30.1
                dns-server 192.168.30.1
                domain-name administration.lab
                lease 86400
                start 192.168.30.2 {
                    stop 192.168.30.254
                }
                static-mapping mech1129 {
                    ip-address 192.168.30.254
                    mac-address xx:xx:xx:xx:xx:xx
                }
            }
        }
    }
    dns {
        forwarding {
            cache-size 150
            listen-on eth1.20
            listen-on eth1.30
            name-server 1.1.1.1
        }
    }
    gui {
        http-port 80
        https-port 443
        listen-address 192.168.10.1
        older-ciphers disable
    }
    nat {
        rule 5000 {
            description "WAN Masquerade"
            log disable
            outbound-interface eth2
            protocol all
            type masquerade
        }
    }
    ssh {
        listen-address 192.168.10.1
        port 22
        protocol-version v2
    }
}
system {
    domain-name infrastructure.lab
    host-name mech2452
    login {
        ...
    }
    name-server 127.0.0.1
    ntp {
        server 0.ubnt.pool.ntp.org {
        }
    }
    offload {
        hwnat disable
        ipsec enable
        ipv4 {
            ...
        }
    }
    syslog {
        ...
    }
    time-zone Pacific/Auckland
}
zone-policy {
    ...
    zone definitions here
    ...
}

Just about to go and check your a) and b) suggestions.

There’s nothing there I can see that shouldn’t work - I suspect you’ll either have a Layer 2 issue (VLANs no working/making it to the router properly) or a firewall/ruleset issue, which as we can’t see your rules we can’t comment on.

Certainly I’d be looking at basic IP connectivity first though.

The client gateway certainly appears to be set properly - the route command has clients on the VLAN20 listed as using 192.168.20.1 so all is well there. In case it is a firewall thing, here are all the rule sets mentioned in the image from the OP. Again, apologies for the length, but there is a lot of repetition.

firewall {
    /* [1] */
    name infrastructure-local {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 200 {
            action accept
            description "Accept HTTP/HTTPS requests"
            destination {
                group {
                    port-group hypertext-ports
                }
            }
            protocol tcp
        }
        rule 600 {
            action accept
            description "Accept DNS requests"
            destination {
                port 53
            }
            protocol tcp_udp
        }
        rule 700 {
            action accept
            description "Accept DHCP requests"
            destination {
                port 67,68
            }
            protocol udp
        }
        rule 800 {
            action accept
            description "Accept SSH requests"
            destination {
                port 22
            }
            protocol tcp
        }
    }
    /* [2] */
    name infrastructure-wan {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 200 {
            action accept
            description "Accept HTTP/HTTPS requests"
            destination {
                group {
                    port-group hypertext-ports
                }
            }
            protocol tcp
        }
    }
    /* [3] */
    name robot-1-local {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 600 {
            action accept
            description "Accept DNS requests"
            destination {
                port 53
            }
            protocol tcp_udp
       }
       rule 700 {
            action accept
            description "Accept DHCP requests"
            destination {
                port 67,68
            }
            protocol udp
        }
    }
    /* [4] */
    name robot-1-wan {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
    }
    /* [9] */
    name administration-robot-1 {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 200 {
            action accept
            description "Accept HTTP/HTTPS requests"
            destination {
                group {
                    port-group hypertext-ports
                }
            }
            protocol tcp
        }
        rule 800 {
            action accept
            description "Accept SSH requests"
            destination {
                port 22
            }
            protocol tcp
        }
    }
    /* [16] */
    name administration-local {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 600 {
            action accept
            description "Accept DNS requests"
            destination {
                port 53
            }
            protocol tcp_udp
        }
        rule 700 {
            action accept
            description "Accept DHCP requests"
            destination {
                port 67,68
            }
            protocol udp
        }
    }
    /* [17] */
    name administration-wan {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 200 {
            action accept
            description "Accept HTTP/HTTPS requests"
            destination {
                group {
                    port-group hypertext-ports
                }
            }
            protocol tcp
        }
    }
    /* [30] */
    name local-infrastructure {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
    }
    /* [31] */
    name local-robot-1 {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
    }
    /* [34] */
    name local-administration {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
    }
    /* [41] */
    name local-wan {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 400 {
            action accept
            description "Accept NTP requests"
            destination {
                port 123 
            }
            protocol udp 
        }
        rule 600 {
            action accept
            description "Accept DNS requests"
            destination {
                port 53
            }
            protocol tcp_udp
        }
    }
    /* [42] */
    name wan-infrastructure {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
    }
    /* [43] */
    name wan-robot-1 {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
    }
    /* [53] */
    name wan-local {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
    }
}

So withouth going over your rules yet - what does the ARP table show. Not interested in the routing table.

show arp

Do you see the MAC Addresses of your clients listed?

What does the ARP table of your clients look like? arp -a on a windows box, ip arp on a Linux box. No idea on a Mac sorry, Google is your friend.

Can a client on the Infrastructure LAN ping Infrastructure’s default GW (192.168.10.1)
Can a client on the Infrastructure LAN ping Robot 1’s default GW (192.168.20.1)

I mean the easy fix here is to temporarily just put in an allow/allow rule. Does ping/routing work then? You know you’ve got a FW rule to fix. If it doesn’t you have to figure you’ve got a Layer 2 issue somewhere.

Hi Tim.

ARP table of the router looks like this:

administrator@mech2452:~$ show arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.10.3             ether   e4:38:83:db:b1:2f   C                     eth1.10
192.168.10.254           ether   f0:de:f1:4c:33:0d   C                     eth1.10
192.168.10.2             ether   f4:92:bf:84:17:23   C                     eth1.10
192.168.20.3             ether   d8:3a:dd:51:d3:30   C                     eth1.22
192.168.20.4             ether   d8:3a:dd:51:d2:07   C                     eth1.22
192.168.30.254           ether   f0:de:f1:4c:33:0d   C                     eth1.30
132.181.61.254           ether   c8:fe:6a:a9:de:00   C                     eth2

So the MAC addresses of the clients on VLAN20 are indeed listed. With regards to your ping queries, did you mean clients on administration VLAN20 pinging default GWs? In either case (infrastructure or administration) all pings are successful.

With regards to the ARP table of a random VLAN20 Raspberri Pi client:

$ arp -a
? (192.168.20.1) at d8:b3:70:3c:be:45 [ether] on eth0

Which looks fine to me as well. I will try your suggestion of a default-action allow rule in the administration-robot-1 ruleset first thing tomorrow.

Ok that’s great then I agree, all L2 stuff is fine - this sounds like it’s going to be just a firewall rule stopping things from working. I will have a look at the config you’ve pasted when I’ve got a bit more time, but yes, a temporary “any/any” rule will help you prove this quickly.

Things are beginning to get interesting. After some experimentation, it appears that the issue is the default-action drop statement in the robot-1 firewall zone definition, ie:

zone-policy {
    ...
    zone robot-1 {
        default-action drop
         /* [9] */
        from administration {
            firewall {
                name administration-robot-1
            }
        }
        ...
        interface eth1.20
    }
    ...
}

If this is changed to default-action accept then I can SSH into a client on VLAN20 without issue. I am somewhat baffled as to why this default action statement over-rides those of firewall ruleset 9 (administration-robot-1) in which SSH traffic is explicitly permitted by rule 800:

firewall {
    ...
    /* [9] */
    name administration-robot-1 {
        default-action drop
        enable-default-log
        rule 1 {
            action accept
            state {
                established enable
                related enable
            }
        }
        rule 2 {
            action drop
            log enable
            state {
                invalid enable
            }
        }
        rule 100 {
            action accept
            description "Accept ICMP requests"
            protocol icmp
        }
        rule 200 {
            action accept
            description "Accept HTTP/HTTPS requests"
            destination {
                group {
                    port-group hypertext-ports
                }
            }
            protocol tcp
        }
        rule 800 {
            action accept
            description "Accept SSH requests"
            destination {
                port 22
            }
            protocol tcp
        }
    }
    ...
}

I will continue to investigate and see what I can find out.

Problem solved (I think). The issue is that there was no robot-1 to administration firewall rule set. These should have appeared in the red box below.

This is clearly an amateur mistake as I have A-B and B-A rule sets and zone policies defined for everything else. Thanks for your help though Tim - it made me think about things slowly and carefully.

1 Like

Glad you figured it out, and thanks for coming back and sharing how you resolved it, might help someone in the future!

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.