Hello
It appears that during boot up/configuration, vyos seems to lose redirect to an ifb.
I use the ifb to apply an inbound traffic policy.
version 1.4-rolling-202301210856
Any suggestions for resolution? Details below…my current workaround is also below…
I do this because I have to mitigate buffer bloat issues as I do a fair amount of video conferencing/voice comms and perhaps my isp sucks… I’m sure I wont be the only to bail on them once fiber is finished (cant wait).
Previous version: 1.4-rolling 202212110827 appears fine, current version 1.4-rolling-202301210856.
Note: during the upgrade, i had to revise the bandwidth from ‘100Mbit’ to ‘100mbit’ as post upgrade/migrate didnt like it, but other than that it seemed like an uneventful upgrade.
Using bmon to check, the queues are most definitely not generated etc before the work around, but after you definitely see the queues created and actual traffic through ifb0.
I also use an outbound policy, which seems unaffected, as I am not using any ifbs for that.
Just for comparison, some testing results 1) post boot, 2) post boot after my work around are below.
My workaround thus far is to (this needs to be done at each boot):
delete interfaces ethernet eth0 redirect 'ifb0'
commit
set interfaces ethernet eth0 redirect 'ifb0'
commit
Things then seem to work fine.
Config snippet of the inbound policy and redirect
vyos@router:~$ show configuration commands | match ifb
set interfaces ethernet eth0 redirect 'ifb0'
set interfaces input ifb0
set qos interface ifb0 egress 'WAN_INGRESS'
vyos@router:~$ show configuration commands | match WAN_INGRESS
set qos interface ifb0 egress 'WAN_INGRESS'
set qos policy shaper WAN_INGRESS bandwidth '100mbit'
set qos policy shaper WAN_INGRESS default bandwidth '100mbit'
set qos policy shaper WAN_INGRESS default burst '15k'
set qos policy shaper WAN_INGRESS default queue-type 'fq-codel'
During the boot i believe it (the redirect) is put into place, but is later removed as the ifb0 interface exists and show a smalls # of packets / bytes. But appears to never be used afterwards.
vyos@router:~$ show interfaces ethernet eth0
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000
link/ether 52:54:00:db:84:2b brd ff:ff:ff:ff:ff:ff
inet XXX.XXX.XXX.XXX/XX brd 255.255.255.255 scope global dynamic eth0
valid_lft 85569sec preferred_lft 85569sec
inet6 fe80::5054:ff:fedb:842b/64 scope link
valid_lft forever preferred_lft forever
Description: WAN Interface
RX: bytes packets errors dropped overrun mcast
3295947834 1934423 0 0 0 0
TX: bytes packets errors dropped carrier collisions
219475315 1197519 0 0 0 0
vyos@router:~$ show interfaces input ifb0
ifb0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN group default qlen 32
link/ether c2:a7:58:9f:c9:81 brd ff:ff:ff:ff:ff:ff
inet6 fe80::c0a7:58ff:fe9f:c981/64 scope link
valid_lft forever preferred_lft forever
RX: bytes packets errors dropped overrun mcast
31675 130 0 0 0 0
TX: bytes packets errors dropped carrier collisions
31675 130 0 0 0 0
show log looking for anything ifb or eth0 returns:
vyos@router:~$ show log | grep -i "ifb\|eth0" | strip-private
Jan 22 09:17:58 kernel: virtio_net virtio0 e2: renamed from eth0
Jan 22 09:18:01 kernel: virtio_net virtio0 eth0: renamed from e2
Jan 22 09:18:13 sudo[2224]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/sh -c /usr/sbin/vyshim VYOS_TAGNODE_VALUE='ifb0' /usr/libexec/vyos/conf_mode/interfaces-input.py
Jan 22 09:18:13 vyos-configd[636]: Received message: {"type": "node", "data": "VYOS_TAGNODE_VALUE=ifb0/usr/libexec/vyos/conf_mode/interfaces-input.py"}
Jan 22 09:18:15 sudo[2265]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/sh -c /usr/sbin/vyshim VYOS_TAGNODE_VALUE='eth0' /usr/libexec/vyos/conf_mode/interfaces-ethernet.py
Jan 22 09:18:15 vyos-configd[636]: Received message: {"type": "node", "data": "VYOS_TAGNODE_VALUE=eth0/usr/libexec/vyos/conf_mode/interfaces-ethernet.py"}
Jan 22 09:18:15 systemd[1]: Starting DHCP client xxxxxx eth0...
Jan 22 09:18:15 systemd[1]: Started DHCP client xxxxxx eth0.
Jan 22 09:18:15 netplugd[995]: eth0: state DOWN flags 0x00001002 BROADCAST,MULTICAST -> 0x00011043 UP,BROADCAST,RUNNING,MULTICAST,10000
Jan 22 09:18:15 netplugd[2325]: /etc/netplug/netplug eth0 in -> pid 2325
Jan 22 09:18:15 dhclient-script-vyos[2308]: Current dhclient xxxxxx 2299, Parent PID: 2293, IP version: 4, All dhclients for interface eth0: 2293 2299
Jan 22 09:18:15 dhclient-script-vyos[2308]: Passing command to /usr/sbin/ip: "link set dev eth0 up"
Jan 22 09:18:15 dhclient[2299]: DHCPDISCOVER on eth0 to xxx.xxx.255.255 port 67 interval 2
Jan 22 09:18:15 netplugd[995]: eth0: state INNING pid 2325 exited status 0
Jan 22 09:18:15 dhclient[2299]: DHCPREQUEST for xxx.xxx.10.141 on eth0 to xxx.xxx.255.255 port 67
Jan 22 09:18:16 dhclient-script-vyos[2432]: Current dhclient xxxxxx 2299, Parent PID: 1, IP version: 4, All dhclients for interface eth0: 2299
Jan 22 09:18:16 dhclient-script-vyos[2432]: Passing command to /usr/sbin/ip: "-4 addr add xxx.xxx.10.141/xxx.xxx.224.0 broadcast xxx.xxx.255.255 valid_lft 3600 preferred_lft 3600 dev eth0 label eth0"
Jan 22 09:18:16 dhclient-script-vyos[2432]: Passing command to /usr/sbin/ip: "link set dev eth0 mtu 1500"
Jan 22 09:18:16 dhclient-script-vyos[2432]: Checking if the route presented in kernel: default via xxx.xxx.0.1 dev eth0 metric 210
Jan 22 09:18:16 dhclient-script-vyos[2432]: Converted vtysh command: "ip route xxx.xxx.0.0/0 xxx.xxx.0.1 eth0 tag 210 210 "
Jan 22 09:18:16 dhclient-script-vyos[2432]: Deleting nameservers with tag "dhcp-eth0" via vyos-hostsd-client
Jan 22 09:18:16 vyos-hostsd[637]: Request data: {"type": "name_servers", "op": "delete", "data": ["dhcp-eth0"]}
Jan 22 09:18:16 dhclient-script-vyos[2432]: Adding nameservers "xxx.xxx.216.1 xxx.xxx.216.2" with tag "dhcp-eth0" via vyos-hostsd-client
Jan 22 09:18:16 vyos-hostsd[637]: Request data: {"type": "name_servers", "op": "add", "data": {"dhcp-eth0": ["xxx.xxx.216.1", "xxx.xxx.216.2"]}}
Jan 22 09:18:18 openvpn-vtun1[2737]: net_route_v4_best_gw result: via xxx.xxx.0.1 dev eth0
Jan 22 09:18:20 dhcpd[3046]: No subnet declaration for ifb0 (no IPv4 addresses).
Jan 22 09:18:20 dhcpd[3046]: ** Ignoring requests on ifb0. If this is not what
Jan 22 09:18:20 dhcpd[3046]: to which interface ifb0 is attached. **
Jan 22 09:18:20 dhcpd[3046]: No subnet declaration for eth0 (xxx.xxx.10.141).
Jan 22 09:18:20 dhcpd[3046]: ** Ignoring requests on eth0. If this is not what
Jan 22 09:18:20 dhcpd[3046]: to which interface eth0 is attached. **
Jan 22 09:18:23 openvpn-vtun1[3241]: net_route_v4_best_gw result: via xxx.xxx.0.1 dev eth0
Testing results from waveform bufferbloat testing…
Post boot
Post boot after work around
Thank you so much for your time and for creating such an awesome product!
Chris