VRRP with latest rolling releases seems to be broken maybe?

I have moved from an earlier VyOS 1.4-rolling-202105050002 version where VRRP was working as expected and I am not sure what or where to from here. Help please :slight_smile:

Just moved recently to VyOS 1.4-rolling-202109280217 and subsequently also to VyOS 1.4-rolling-202110180513 when I noticed a problem with my VRRP and thought Id check if its working in latest.

The interfaces do not appear to be assigned correctly and the script configuration is not loaded up at all and so any VRRP scripts do not run. Also errors in journalctl and note the output from show interfaces and show interfaces vrrp

journalctl | grep keepalived on the 2 latest mentioned releases

(standard input):Oct 19 14:20:06 vyos007 Keepalived[87492]: Command line: '/usr/sbin/keepalived' '--use-file' '/run/keepalived/keepalived.conf' '--pid'
(standard input):Oct 19 14:20:06 vyos007 Keepalived[87492]:               '/run/keepalived/keepalived.pid' '--dont-fork' '--snmp'
(standard input):Oct 19 14:20:06 vyos007 Keepalived[87492]: Opening file '/run/keepalived/keepalived.conf'.
(standard input):Oct 19 14:20:06 vyos007 Keepalived[87492]: NOTICE: setting config option max_auto_priority should result in better keepalived performance
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: Opening file '/run/keepalived/keepalived.conf'.
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 30) (cam): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 46) (dmz): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 62) (download): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 78) (guest): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 94) (iot): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 110) (lan): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 126) (mgmt): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 Keepalived_vrrp[87495]: (/run/keepalived/keepalived.conf: Line 142) (public): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: Starting FIFO pipe for Keepalived
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: Unable to load configuration:
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: PIPE already exist: /run/keepalived/keepalived_notify_fifo
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: Message reading start
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: Message processing start
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: Received message: GROUP "sync" BACKUP 0
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: GROUP sync changed state to BACKUP
(standard input):Oct 19 14:20:06 vyos007 keepalived-fifo.py[87501]: Error processing message: 'KeepalivedFifo' object has no attribute 'vrrp_config'
(standard input):Oct 19 14:20:11 vyos007 keepalived-fifo.py[87501]: Received message: INSTANCE "cam" BACKUP 254
(standard input):Oct 19 14:20:11 vyos007 keepalived-fifo.py[87501]: INSTANCE cam changed state to BACKUP
(standard input):Oct 19 14:20:11 vyos007 keepalived-fifo.py[87501]: Error processing message: 'KeepalivedFifo' object has no attribute 'vrrp_config'

this shows the interfaces as a result just tacked onto the existing vifs instead of their own VRRP interfaces

mario@vyos007# run show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             -                                 u/u
eth0.7           192.168.7.252/24                  u/u  Management
                 192.168.7.253/24
eth0.11          192.168.11.252/24                 u/u  IOT
                 192.168.11.253/24
eth0.13          192.168.13.252/24                 u/u  LAN
                 192.168.13.253/24
eth0.17          192.168.17.252/24                 u/u  Public
                 192.168.17.253/24
eth0.53          192.168.53.252/24                 u/u  Cam
                 192.168.53.253/24
eth0.67          192.168.67.252/24                 u/u  DMZ
                 192.168.67.253/24
eth0.79          192.168.79.252/24                 u/u  Download
                 192.168.79.253/24
eth0.131         192.168.131.252/24                u/u  Guest
                 192.168.131.253/24
eth0.167         removed                   u/u  WAN
eth0.197         -                                 u/u  WAN_BCK
lo               127.0.0.1/8                       u/u
                 ::1/128
[edit]
mario@vyos007#

This is usually full of VRRP interfaces on their own “eth0.nnvnn” example interfaces

mario@vyos007# run show interfaces vrrp
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
[edit]

A partial bit of VRRP config with much of the same other few vlans removed

mario@vyos007# show high-availability
 vrrp {
     group cam {
         address 192.168.53.253/24
         advertise-interval 1
         description Cam
         hello-source-address 192.168.53.252
         interface eth0.53
         peer-address 192.168.53.254
         preempt-delay 3
         priority 254
         rfc3768-compatibility
         vrid 53
     }
     group lan {
         address 192.168.13.253/24
         advertise-interval 1
         description LAN
         hello-source-address 192.168.13.252
         interface eth0.13
         peer-address 192.168.13.254
         preempt-delay 3
         priority 254
         rfc3768-compatibility
         vrid 13
     }
.....
..... #more vrrp interfaces here in same style as the first ones 
.....
     sync-group sync {
         member cam
         member guest
         member mgmt
         member lan
         member iot
         member public
         member dmz
         member download
         transition-script {
             backup "/config/scripts/vrrp-trans-fail.sh backup"
             fault "/config/scripts/vrrp-trans-fail.sh backup"
             master "/config/scripts/vrrp-trans-master.sh master"
             stop "/config/scripts/vrrp-trans-fail.sh backup"
         }
     }
 }

I finally looked in the “new?” /run/keepalived/keepalived.conf file and see at the bottom there is no transition scripts, the other parts look similar to VyOS 1.4-rolling-202105050002

vrrp_sync_group sync {
    group {
        cam
        guest
        mgmt
        lan
        iot
        public
        dmz
        download
    }
}

And in the old VyOS 1.4-rolling-202105050002 from /etc/keepalived/keepalived.conf

vrrp_sync_group sync {
       group {
                            cam
                            guest
                            mgmt
                            lan
                            iot
                            public
                            dmz
                            download
                    }

                    notify_master "/opt/vyatta/sbin/vyatta-vrrp-conntracksync.sh master sync"
            notify_backup "/opt/vyatta/sbin/vyatta-vrrp-conntracksync.sh backup sync"
            notify_fault "/opt/vyatta/sbin/vyatta-vrrp-conntracksync.sh fault sync"
        }

Ok, I now realise the very bottom part is not my transition scripts at all, but still, the problems remain and are valid

As far as I understand from logs, it says that you can’t use rfc3768-compatibility when using unicast communication between vrrp-peers. Is it possible for you to use multicast?

It would be possible but I have not had the time to change the configuration yet.

I am curious tho as it has been working for about 3 or 4 years that I have been on vyos at home, including in the may 1.4 release

It seems some upstream “keepalived” issue
It doesn’t work even in 1.3.0-rc6
Keepalived log

Oct 19 11:54:29 r4-epa1 Keepalived_vrrp[2477]: (/etc/keepalived/keepalived.conf: Line 38) (GRP02): Cannot use VMAC/ipvlan with unicast peers - clearing use_vmac

And possible it was fixed here https://github.com/acassen/keepalived/commit/97429b3b7e6ec2f5b9c93a5d507b152bab30f919

in keepalived 2.2.x

I created a task T3914

Haha I thought my mind was having at me, thats a good find.
Thanks for the task also, I was not sure if I should create or not but that settles it.

Cheers!