Adding macsec interface to bridge

Ralf · August 10, 2020, 12:41pm

I tried to get macsec to work on layer2, tried adding macsec interface to bridge and this is the output:

vyos@vyos13-1# set interfaces bridge br0 member interface eth1
[edit]
vyos@vyos13-1# commit
[ interfaces bridge br0 ]
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):

Make sure you are running the latest version of the code available at
Consult the forum to see how to handle this issue
Join our community on slack where our users exchange help and advice

When reporting problems, please include as much information as possible:

do not obfuscate any data (feel free to contact us privately if your
business policy requires it)
and include all the information presented below

Report Time: 2020-08-10 14:18:45
Image Version: VyOS 1.3-rolling-202008100118
Release Train: equuleus

Built by: [email protected]
Built on: Mon 10 Aug 2020 01:18 UTC
Build UUID: d83021b4-7620-4a8f-9166-f466497ae3dc
Build Commit ID: 9e92985ad38423

Architecture: x86_64
Boot via: installed image
System type: VMware guest

Hardware vendor: VMware, Inc.
Hardware model: VMware Virtual Platform
Hardware S/N: VMware-56 4d 90 d1 ae cf ed 67-55 1c c0 73 dc d6 85 16
Hardware UUID: d1904d56-cfae-67ed-551c-c073dcd68516

Traceback (most recent call last):
File “/usr/libexec/vyos/conf_mode/interfaces-bridge.py”, line 136, in
apply(c)
File “/usr/libexec/vyos/conf_mode/interfaces-bridge.py”, line 127, in apply
br.update(bridge)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/bridge.py”, line 245, in update
self.add_port(interface)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/bridge.py”, line 180, in add_port
return self.set_interface(‘add_port’, interface)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/control.py”, line 184, in set_interface
return self._set_command(self.config, name, value)
File “/usr/lib/python3/dist-packages/vyos/ifconfig/control.py”, line 109, in _set_command
return self._command_set[name].get(‘format’, lambda _: _)(self._cmd(cmd))
File “/usr/lib/python3/dist-packages/vyos/ifconfig/control.py”, line 51, in _cmd
return cmd(command, self.debug)
File “/usr/lib/python3/dist-packages/vyos/util.py”, line 179, in cmd
raise OSError(code, feedback)
FileNotFoundError: [Errno 2] failed to run command: ip link set dev eth1 master br0
returned:
exit code: 2

noteworthy:
cmd ‘ip link set dev eth1 master br0’
returned (out):

returned (err):
RTNETLINK answers: Device or resource busy

[[interfaces bridge br0]] failed
Commit failed
[edit]
vyos@vyos13-1#

vyos macsec config from wiki:
set interfaces macsec macsec1 address ‘192.0.2.1/24’
set interfaces macsec macsec1 security cipher ‘gcm-aes-128’
set interfaces macsec macsec1 security encrypt
set interfaces macsec macsec1 security mka cak ‘232e44b7fda6f8e2d88a07bf78a7aff4’
set interfaces macsec macsec1 security mka ckn ‘40916f4b23e3d548ad27eedd2d10c6f98c2d21684699647d63d41b500dfe8836’
set interfaces macsec macsec1 source-interface ‘eth1’

rob · August 11, 2020, 5:21pm

Hey,

Thanks for reporting this error, i will create a vyos phabricator task about it.

to add a macsec interface to a brigde, remove the ip addresses from the macsec interface and add it to the bridge not the base interface.

set interface bridge br0 member interface macsec1

also set the ip addresses back to the bridge interface. Worked in my lab.

rob · August 11, 2020, 5:40pm

https://phabricator.vyos.net/T2788

Ralf · August 12, 2020, 8:18am

Thank you for replay, I didn’t even consider adding macsec1 do the bridge.

I configured a bridge on both sides of the macsec:

vyos@vyos13-1:~$ sh bridge br0
bridge name bridge id STP enabled interfaces
br0 8000.000c29d68520 yes eth2
macsec1

vyos@vyos13-2:~$ sh bridge br0
bridge name bridge id STP enabled interfaces
br0 8000.000c29c90321 yes eth2
macsec1

On eth2 there is end host connected, unfortunately there is no communication through macsec.
Host1 10.10.123.4 connected to vyos13-1
Host2 10.10.123.3 connected to vyos13-2

When I run ping from host1 connected to vyos13-1, I don’t see the ping on other side of the macsec.

Tcpdump on vyos13-1 port eth2 show arp requests:
09:58:51.521443 ARP, Request who-has 10.10.123.3 tell 10.10.123.4, length 46

But there is nothing on vyos13-2 port eth2.

I cannot add the ip address back on macsec1:

vyos@vyos13-1# set interfaces macsec macsec1 address ‘192.0.2.1/24’
[edit]
vyos@vyos13-1# commit
[ interfaces macsec macsec1 ]
Cannot assign address to interface “macsec1” as it is a member of bridge “br0”!

[[interfaces macsec macsec1]] failed
Commit failed
[edit]

rob · August 12, 2020, 4:16pm

Hi,

i wrote this a little bit confusing on my first post, earse the word “back”
The ip address must configure on the bridge. A Member interface are not able to hold an address.

Ralf · August 17, 2020, 9:05am

I added the ip on bridge interface, I can ping from vyos13-1 to vyso13-2, tcpdump shows packets on macsec interface on both sides.

Communication from connected hosts on layer2 does not work.

rob · August 17, 2020, 6:34pm

can you draw the topology? so i can try this in my lab.

Ralf · August 18, 2020, 7:22am

vyos13-1.txt (1.4 KB) vyos13-2.txt (1.4 KB) topology

rob · August 18, 2020, 5:48pm

i build this in EVE-ng and it worked:

the terminal is on VyOS1 as a client an it can ping all other VyOS host. VyOS2 and 3 take care of the macsec connection.

VyOS2:

set interfaces bridge br0 address '10.10.10.2/24'
set interfaces bridge br0 member interface eth1
set interfaces bridge br0 member interface macsec1
set interfaces ethernet eth0 hw-id '50:00:00:02:00:00'
set interfaces ethernet eth1 hw-id '50:00:00:02:00:01'
set interfaces loopback lo
set interfaces macsec macsec1 security cipher 'gcm-aes-128'
set interfaces macsec macsec1 security encrypt
set interfaces macsec macsec1 security mka cak '232e44b7fda6f8e2d88a07bf78a7aff4'
set interfaces macsec macsec1 security mka ckn '40916f4b23e3d548ad27eedd2d10c6f98c2d21684699647d63d41b500dfe8836'
set interfaces macsec macsec1 source-interface 'eth0'

VyOS3:

set interfaces bridge br0 address '10.10.10.3/24'
set interfaces bridge br0 member interface eth1
set interfaces bridge br0 member interface macsec1
set interfaces ethernet eth0 hw-id '50:00:00:03:00:00'
set interfaces ethernet eth1 hw-id '50:00:00:03:00:01'
set interfaces loopback lo
set interfaces macsec macsec1 security cipher 'gcm-aes-128'
set interfaces macsec macsec1 security encrypt
set interfaces macsec macsec1 security mka cak '232e44b7fda6f8e2d88a07bf78a7aff4'
set interfaces macsec macsec1 security mka ckn '40916f4b23e3d548ad27eedd2d10c6f98c2d21684699647d63d41b500dfe8836'
set interfaces macsec macsec1 source-interface 'eth0'

Your hosts are in the same subnet?
Are you able to ping the br interface of each VyOS Router (if there in same address space)
When you try to connect from host1 to host2 did you see traffic on the interfaces eth2 and eth1 of both vyos routers?

Ralf · August 21, 2020, 12:51pm

In my virtual setup I could not ping the br interface. They are in the same subnet. In my previous post I wrote that I don’t see traffic on macsec tunnel. I see arp request on vyos13-1, but nothing on vyos13-2.

I redid this setup on hardware devices and it works, my end host communicate on layer2 through macsec. Seems there is some misconfiguration in my virtual setup. Thank you for confirming that it should work.

system · August 25, 2020, 5:53pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

c-po · September 21, 2020, 4:20pm

The error message is of course not good. But the error is obvious. You can not add the source interface of a macsec channel to a bridge. You can add the macsec interface to a bridge of course.

All further tracking is done via ⚓ T2788 adding interface with macsec to bridge failded