Can't create a virtual Switch in proxmox

General questions
1

Hello,
I am trying to set up a virtual switch (vyos based) on proxmox.

On proxmox I have created two “linux bridge” vmbr2 and vmbr3 and I have assigned both to a vyos VM and one of each to two Debian 12 VM.

on the Debian VM i have configure the two virtual ethernet port like this :

enp6s19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether b2:4d:17:d0:26:ea brd ff:ff:ff:ff:ff:ff
inet 192.168.1.10/24 brd 192.168.1.255 scope global enp6s19

and

enp6s19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 2a:9c:61:62:40:99 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.20/24 brd 192.168.1.255 scope global enp6s19

On the vyos VM I have created a bridge ad assigned the two port to the bridge

But the two linux instance can’t talk to each other:

root@debian-test:~# ping 192.168.1.10
PING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.
From 192.168.1.20 icmp_seq=1 Destination Host Unreachable
From 192.168.1.20 icmp_seq=2 Destination Host Unreachable
From 192.168.1.20 icmp_seq=3 Destination Host Unreachable

I just notice that the 2 “linux bridge” are Admin Down, but no idea on how to change them to Up:

Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description

br0 192.168.1.1/24 u/D
eth0 - A/D
eth1 - A/D
eth2 - A/D
lo 127.0.0.1/8 u/u
::1/128

Any idea ?
Thanks
Albert

2

Can you share your configuration?

I’m assuming you have the member interfaces defined?

https://docs.vyos.io/en/equuleus/configuration/interfaces/bridge.html

set interfaces bridge <interface> member interface <member>
3

To share the configuration of VyOS:

show config commands | strip-private

It would also be handy to get the config of your VM-guest in Proxmox.

That is stored in a file with the VM-guest id in (I think it is):

/etc/pve/qemu-server

If its not there do a:

sudo find / -iname “qemu-server”

to locate that directory.

4

Thanks for your answers.!
Here is my conf:
set interfaces bridge br0 address ‘xxx.xxx.1.1/24’
set interfaces bridge br0 member interface eth1
set interfaces bridge br0 member interface eth2
set interfaces ethernet eth0 address ‘dhcp’
set interfaces loopback lo
set system config-management commit-revisions ‘100’
set system conntrack modules ftp
set system conntrack modules h323
set system conntrack modules nfs
set system conntrack modules pptp
set system conntrack modules sip
set system conntrack modules sqlnet
set system conntrack modules tftp
set system console device ttyS0 speed ‘115200’
set system host-name xxxxxx
set system login user xxxxxx authentication encrypted-password xxxxxx
set system login user xxxxxx authentication plaintext-password xxxxxx
set system ntp server xxxxx.tld
set system ntp server xxxxx.tld
set system ntp server xxxxx.tld
set system syslog global facility all level ‘info’
set system syslog global facility protocols level ‘debug’

And here is the VM instance conf in proxmox:
balloon: 0
boot: order=scsi0
cores: 1
cpu: x86-64-v2-AES
memory: 1024
meta: creation-qemu=8.0.2,ctime=1697657161
name: vyos
net0: virtio=FE:4E:07:40:68:5D,bridge=vmbr0
net1: virtio=6E:EF:4F:40:01:B0,bridge=vmbr2
net2: virtio=96:CE:94:64:5A:0D,bridge=vmbr3
numa: 0
ostype: l26
parent: Just_installed
scsi0: local-lvm:vm-100-disk-0,iothread=1,size=4G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=5e96f884-45d8-4ea2-8f1b-3910369fea59
sockets: 1
vmgenid: 43e2c380-391a-47c5-8c4d-8905b39a7b05

[Just_installed]
#just installed pwd test
balloon: 0
boot: order=scsi0
cores: 1
cpu: x86-64-v2-AES
memory: 1024
meta: creation-qemu=8.0.2,ctime=1697657161
name: vyos
net0: virtio=FE:4E:07:40:68:5D,bridge=vmbr0
net1: virtio=6E:EF:4F:40:01:B0,bridge=vmbr2
net2: virtio=96:CE:94:64:5A:0D,bridge=vmbr3
numa: 0
ostype: l26
scsi0: local-lvm:vm-100-disk-0,iothread=1,size=4G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=5e96f884-45d8-4ea2-8f1b-3910369fea59
snaptime: 1697893874
sockets: 1
vmgenid: 240c8b4e-ea64-48e9-8099-8be1f54e06d5

5

try to add any config to interfaces
For example description, it should trigger to up interface

6

Thanks! that did the trick!
Is working now.

7

Other tips regarding VM-guest settings in Proxmox:

  1. Unless you run your Proxmox in a cluster set CPU type to “host”.

  2. Enable at least 2 VCPU (1 socket) or more if you can afford.

VCPU are basically threads on the VM host, a rule of thumb is that you should be able to have at least 1.5x number of threads your CPU’s can deal with at once but not more VCPU for a single VM-guest than what the total number of threads are available. Example 32 core 64 threads CPU should not set VCPU for a single VM guest to more than 64 but the total amount (if you sum up all running VM-guests) can be in this example 1.5*64=96 VCPU in total.

  1. Optional if you want to enable NUMA (I do it by default - doesnt hurt and if/when its needed its good for the VM-guest to have insight in the CPU core and cache distribution to run threads as efficient as possible).

  2. Set system type to Q35.

  3. In storage settings enable discard.

  4. Also good that you already disabled balooning for the RAM.

  5. 1GB RAM should be fine but as with VCPU if you can afford more then you should give more for VyOS (depends also what kind of services and number of concurrent sessoins your VyOS will deal with). A rule of thumb for conntrack is that it will use approx 320 bytes per session/flow.

  6. Disable “USB Tablet pointer”.

  7. Dont forget to enable autostart (so the VM-guest starts automatically) but also set a starorder and start vs shutdown timeout.

Start timeout is amount of seconds between this VM-guest vs next VM-guest when autostarting and shutdown timeout is how long Proxmox should wait for a VM-guest after shutdown have been given before it will just “cut the power” to the VM-guest. I often set 10 seconds between VM-guests for startup and 600 seconds as shutdown timeout (if it shutdown on its own before this then the timeout will never trigger).

When it comes to VyOS you probably want to have it to start just after the NTP server (if you have such as VM-guest) have started. And then measure how long it takes for VyOS to get operational (you will see this in the console - my VyOS in the lab takes approx 60 seconds to boot but longer with more routes and firewall rules) and put that as start timeout on the next VM-guest.

Example:

VM-101: NTP, Startorder: 101, Start timeout: 10, Shutdown timeout: 600
VM-102: VyOS, Startorder: 102, Start timeout: 10, Shutdown timeout: 600
VM-103: DNS, Startorder: 103, Start timeout: 60, Shutdown timeout: 600
VM-104: WEB, Startorder: 104, Start timeout: 10, Shutdown timeout: 600
…

1 Like
8

Thanks , I will take in consideration your suggestion when I will move to “production” , for now I just making some test to better understand if what I want to do is feasible.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.