Connection instability from bridging a wired network with an WIFI network

General questions
, ,
1

Hi everyone,

I’m a bit lost here and I would hopping that someone might be able to identify what is going on in my network.

I’ve recently moved to a new fresh installation of VyOS 1.4, running on a new machine.
Since I’m using VyOS as my home router and the new machine is plenty powerful, I’m experimenting not having a switch for the home devices and having the devices connected directly to VyOS.
To achieve this I simple installed several NICs.

This is part of my interfaces configuration, but I’m focusing on the issue I’m having:

  • eth0: WAN, DHCP
  • eth1: LAN Untrusted (10.0.1.1/24)
    • vif 10: IOT (10.0.3.1/24)
    • vif 20: LAN Trusted (10.0.2.1/24)
  • eth2: IOT (10.0.3.1/24, currently disconnected)
  • eth3: LAN Trusted (10.0.2.1/24)

Eth1 network cable essentially comes from an Ubiquity AC LR Access Point, which will send untagged traffic from Guest SSID, Vlan 10 from the IOT SSID and Vlan 20 for the Trusted SSID.
VyOS does not have an Wifi card installed.

As you can see, eth1.20 and eth3 share the same IP address.
This is causing devices connected to the Trusted Wi-Fi network to not have internet access and for the connection to be very unstable. (they keep reconnecting every couple seconds)

Funny enough, IOT devices connected to the IOT SSID are working fine, but then I realized that I don’t have any device connected on the wired NIC. However, in the LAN Trusted I do, I have a Desktop computer.
On top of that, I even tested connecting my phone to the IOT SSID and works fine.
Additionally, connecting to the GUEST SSID also works fine, so the issue is with the Trusted SSID.

To troubleshoot further, wondering if the issue was regarding having 2 interfaces with the same IP, I changed them, where eth1.20 now had 10.0.22.1/24. Created its dedicated DHCP config and everything started working fine, but I would like to have them share the same subnet.

Next, I tried bridging the 2 interfaces, wondering if was some weird thing like VyOS not knowing where to send the packets, etc. Create the bridge, assign eth3 and eth1.20 as their members, remove the address from eth1.20 and eth3. Set now the address 10.0.2.1/24 on the bridge and immediately, the same issue starts to occur again.

I wonder if there is some broadcast issue or something, I’m a bit lost on what might be causing this.

Any ideas?

Thanks.

2

Yes, you should have a different IP range. You can’t have one IP address associated with multiple MAC addresses. Otherwise the forwarding table is all messed up/not predictable and you’d experience the symptoms you are describing.

3

You can use VRFs if you need to use the same subnet for multiple different networks: VRF — VyOS 1.5.x (circinus) documentation

4

VRFs create different routing tables. It sounds like the original poster wanted to bridge an untagged interface and a q tagged interface like this: Bridge — VyOS 1.3.x (equuleus) documentation

Ralm, did you move the IP address from the Ethernet interface to the bridge interface?

5

Yes, my intention was to have VyOS act as both a Switch and a Router at the same time in a way.

Previously, I had a switch besides VyOS that would aggregate multiple cables into several VLANs + Untagged traffic, those would enter VyOS on a single 10GbE interface.
The VLANs were used mostly just to add a separation to the subnets.

Now, I’m trying to remove the switch that was right besides VyOS doing this.
Ideally I would have a single interface per subnet, however due to my physical connections, that is not really possible.

This diagram is not exact, it’s a simplification to transmit the overall idea regarding my previous setup and what I’m trying to change.

image
image1734×1041 56.7 KB

The number of ports used or the number of devices on the main aggregation switch (the one marked with the X), is not exact.

What I’m trying to do is eliminate this aggregation switch and have the devices and switches connected directly.
VyOS went from having just a single 10GbE port, to 2x 10GbE and 6x Gbit.

To facilitate the DHCP server pools and firewall rules, is the reason I was trying to keep a single subnet for each Vlan, which have devices across the multiple devices.

Regarding your question, yes, I did move the IP address from both Ethernet interfaces to the Bridge and had the issue anyway.
This is an aspect that is partially confusing to me, should I just leave the Bridge with no IP and have a single member with it?

Any guidance would be appreciated.

6

No, the bridge interface should have the IP. You want the config to be something like:

set interfaces bridge br114 address ‘10.97.114.254/24’
set interfaces bridge br114 member interface eth1.114
set interfaces bridge br114 member interface eth2
set interfaces ethernet eth0 address ‘dhcp’
set interfaces ethernet eth0 hw-id ‘00:04:24:0e:0d:1c’
set interfaces ethernet eth1 hw-id ‘00:04:24:0f:87:5a’
set interfaces ethernet eth1 vif 114
set interfaces ethernet eth2 hw-id ‘00:04:24:10:2b:79’
set interfaces loopback lo
set nat source rule 10 outbound-interface name ‘eth0’
set nat source rule 10 protocol ‘all’
set nat source rule 10 source address ‘10.97.114.0/24’
set nat source rule 10 translation address ‘masquerade’
set service dhcp-server shared-network-name test1 subnet 10.97.114.0/24 default-router ‘10.97.114.254’
set service dhcp-server shared-network-name test1 subnet 10.97.114.0/24 name-server ‘10.8.8.8’
set service dhcp-server shared-network-name test1 subnet 10.97.114.0/24 range 1 start ‘10.97.114.4’
set service dhcp-server shared-network-name test1 subnet 10.97.114.0/24 range 1 stop ‘10.97.114.99’

In this example, you’ll see that eth1 is like your trunked interface to the wifi and eth2 is the untagged interface paired with eth1 qtag 114. Then run “show interfaces” and you’ll see a mac address associated with br114 in addition to the IP address of interest (in this case 10.97.114.254):

vyos@vyos:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address MAC VRF MTU S/L Description

br114 10.97.114.254/24 16:07:a4:9a:c9:d7 default 1500 u/u
eth1 - 00:04:24:0f:87:5a default 1500 u/u
eth1.114 - 00:04:24:0f:87:5a default 1500 u/u
eth2 - 00:04:24:10:2b:79 default 1500 u/u

On your devices, when you run “show arp”, you should expect to see the MAC address associated with your bridge (in this example, 16:07:a4:9a:c9:d7) for your bridge interface/default router IP (in this example, 10.97.114.254). If your devices see the MAC as something other than the bridge interface MAC, something is wrong so that is the first thing to check.

Dave

7

Hi Dave,

Yes, this was what essentially I’ve tried previously but wasn’t working.
However, you sent me to the right track.

I just tried creating the bridge again, when I did, immediately the wifi devices started all having connectivity issues.
But you mentioning the ARP tables, etc let me wondering “is VyOS even updating these”, so I decided to reboot it after I commited the changes.
Looking at the “show arp” output between before doing the interfaces commit change, after the commit and after the reboot, I believe that VyOS in a way, did update things on its side, but possibly something else didn’t or some other “service” or “component” within VyOS, such as routing might have not.

Still, after the reboot it came back up, everything was working fine, now using the bridge.
To be clear, only VyOS was rebooted and was the only difference vs what I’ve tried before, so I still think that something within VyOS is not refreshing or whatever, when you create a bridge.

For the sake of detail in case someone is interested in investigating this further, the list of changes that I’ve performed to repro this was:

set interfaces bridge br10 description "Trusted"
delete interfaces ethernet eth1 vif 10 address
delete interfaces ethernet eth3 address
delete interfaces ethernet eth6 address
set interfaces bridge br10 address '10.0.2.1/24'
set interfaces bridge br10 member interface eth1.50
set interfaces bridge br10 member interface eth3
set interfaces bridge br10 member interface eth6

Before this, eth1.50 had address 10.0.22.1/24’, eth3 had 10.0.2.1/24 and eth6 had 10.0.23.1/24.

Well, in the end, I’m happy that this is working and I can continue setting up the network with the new setup :slight_smile:

Thank you.

1 Like
8

oh @davespc, I would like to take this opportunity to ask one thing.

How does it work when it comes with bridged interfaces, that are using different MTUs?
For example, in my setup, that eth6 interface is a 10GbE interface, with MTU set to 9000, the others are set to the normal 1500.
The MTU is set on an interface level and no MTU defined on the bridge.

Would love your insights on this.
Thanks in advance.

9

@Ralm, as you said MTU is per interface, not per bridge (just like physical switches). I would recommend keeping the default MTU on your end devices unless you are sure that the end-to-end path for all communication will be something > 1500 (maybe this is a NAS/SAN only network or something). Otherwise, any efficiency benefit from a higher MTU will be more than offset by the CPU overhead you incur by having to deal with IP fragments. Modern NICs have hardware large send offload (LSO) and large receive offload (LRO) which take care of chunking packets into 1500 bytes and take that off your CPU. My $0.02.

10

yes, the reason for the 10GbE is due to being a NAS and because at those speeds I do notice a bit of a difference on large transfers.

From some research, I did come to the realization that at worse, it would just result on fragmentation.

I will need to run some tests, in better understanding this relationship between the interfaces, since currently the Bridge used the default 1500 for MTU.

thanks