ARP? Traffic being dropped between VyOS and Container

Full firewall config below. I’ve configured drop invalid state, as I think is standard for all VyOS deployments. Unfortunately this is resulting in my Tailscale container not working. I pretty much followed this guide and it works perfectly with the invalid state rule removed. But with the rule, I’m constantly getting this error and the docker isn’t able to do anything. The container logs show it can’t even perform DNS lookups and in the system log:

kernel: [STATE-POLICY-INV-D]IN= OUT=veth0 ARP HTYPE=1 PTYPE=0x0800 OPCODE=2 MACSRC=22:e0:7f:22:3f:32 IPSRC=10.3.4.33 MACDST=8e:5a:e6:c8:d5:02 IPDST=10.3.4.36

A few questions:
Is this actually ARP traffic? Why is that being filtered out? I’m surprised because there isn’t a protocol ARP to configure in any rules.
Why isn’t this being allowed by other rules?
I’m still fairly new to Docker/Podman/containers. I configured the container network with 10.3.4.32/28. is .33 the gateway basically?
Any ideas what can be done to fix this? I don’t think I should go connecting a router to the internet that allows invalid state.

Thanks!

Firewall section config:

 global-options {
     state-policy {
         established {
             action accept
         }
         invalid {
             action drop
             log
             log-level info
         }
         related {
             action accept
         }
     }
 }
 group {
     address-group hass {
         address 10.3.3.15
         description "Home assistant exceptions"
     }
     address-group sonos {
         address 10.3.5.92
         address 10.3.5.96
         address 10.3.5.124
         description "Home assistant exceptions"
     }
     network-group internal {
         network 100.64.0.0/10
         network 10.0.0.0/8
         network 192.168.0.0/16
         network 172.16.0.0/12
     }
     network-group nat {
         network 192.168.1.0/24
         network 10.3.4.32/28
         network 10.3.3.0/24
         network 10.3.7.0/28
         network 10.3.5.0/24
         network 10.3.7.64/27
         network 10.3.6.0/24
         network 10.3.7.16/28
     }
     network-group services {
         network 10.3.3.0/24
         network 10.4.3.0/24
     }
     network-group trusted {
         network 100.64.0.0/10
         network 10.4.1.0/27
         network 10.4.1.48/28
         network 10.4.1.32/28
         network 10.4.1.64/28
         network 10.3.4.0/27
         network 10.3.7.0/28
         network 10.3.7.16/28
         network 10.3.3.0/24
         network 10.4.3.0/24
         network 10.3.4.32/28
     }
     port-group hass {
         description "Home assistant exceptions for sonos"
         port 1443
         port 1400
         port 8123
     }
     port-group public-services {
         port 53
         port 123
         port 80
         port 443
     }
 }
 ipv4 {
     forward {
         filter {
             default-action drop
             rule 100 {
                 action accept
                 description "Allow trusted networks to all destinations"
                 source {
                     group {
                         network-group trusted
                     }
                 }
             }
             rule 101 {
                 action accept
                 description "Allow internet access"
                 destination {
                     group {
                         network-group !internal
                     }
                 }
                 protocol all
             }
             rule 200 {
                 action accept
                 description "Allow services internally"
                 destination {
                     group {
                         network-group services
                         port-group public-services
                     }
                 }
                 protocol tcp_udp
                 source {
                     group {
                         network-group internal
                     }
                 }
             }
             rule 300 {
                 action accept
                 description "Allow ICMP to service networks"
                 destination {
                     group {
                         network-group services
                     }
                 }
                 protocol icmp
             }
             rule 400 {
                 action accept
                 description "Home assistant exceptions for sonos"
                 destination {
                     group {
                         address-group hass
                         port-group hass
                     }
                 }
                 protocol tcp_udp
                 source {
                     group {
                         address-group sonos
                     }
                 }
             }
         }
     }
     input {
         filter {
             default-action drop
             rule 10 {
                 action accept
                 destination {
                     port ssh
                 }
                 inbound-interface {
                     name eth1
                 }
                 protocol tcp_udp
                 source {
                     address 192.168.1.0/24
                 }
             }
             rule 20 {
                 action accept
                 description "Allow tailscale container in"
                 source {
                     address 10.3.4.32/28
                 }
             }
         }
     }
     output {
         filter {
             default-action accept
         }
     }
     prerouting {
     }
 }

Find global option for bridged traffic

Other thing to look at and test is promiscious mode for the physical nic which your VM/CT will be using.

Not uncommon that this is needed for the kernel filters to not drop the replies.

If this is the case it can be verified that the VM/CT can only ping the physical IP of the host but not reach outside of it.

When doing tcpdump on the host you will see that the VM/CT sends a “arp whohas?” which is sent to the destination, reply then returns from lets say the default gateway but VM/CT never gets it so another second or so later you see another “arp whohas?” for the same IPv4 from the same VM/CT.

Thanks! The solution was right under my nose, of course. I just didn’t realize this was bridged traffic.

vyos@tmperouter02# set firewall global-options apply-to-bridged-traffic 
Possible completions:
   invalid-connections  Accept ARP and DHCP despite they are marked as invalid
                        connection

I have ALOT of learning to do but I love how VyOS allows you to extend the functionality with containers while not breaking the image based package management structure or breaking out of the configuration management system!

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.