VRRP transition scripts not running VyOS commands?

Hi all, for some reason I am finding my vrrp transition scripts are not running.

I have spent a bit of time trying different styles and logging in script.
Logging worked, so scripts are executing at least.

Just the VyOS commands that I would like the script to run are not executing…
Can someone please see what may actually be happening, the documentation seems clear and even some of the examples around the forums and/or internet seem similar in basic layout.

NONE of the below commands execute either when i run ‘restart vrrp’ or restart the entire router.
Please note I am not seeing any references that the transition scripts are running from ‘show log’ and I swear there used to be a reference to “Running script” when VRRP state changed.

Thanks for your time!

Current simple form

mario@vyos009# show high-availability vrrp group lan
advertise-interval 1
description LAN
hello-source-address 192.168.13.254
interface eth1.13
peer-address 192.168.13.252
preempt-delay 3
priority 250
rfc3768-compatibility
transition-script {
backup /config/scripts/vrrp-trans-fail.sh
fault /config/scripts/vrrp-trans-fail.sh
master /config/scripts/vrrp-trans-master.sh
stop /config/scripts/vrrp-trans-fail.sh
}
virtual-address 192.168.13.253/24
vrid 13
[edit]

/config/scripts/vrrp-trans-master.sh

mario@vyos009# cat /config/scripts/vrrp-trans-master.sh
#!/bin/vbash
source /opt/vyatta/etc/functions/script-template

configure
delete interfaces ethernet eth0 vif 167 address 192.168.167.167/24
set interfaces ethernet eth0 vif 167 address dhcp
delete nat destination rule 200 disable
delete nat destination rule 201 disable
delete nat destination rule 399 disable
delete nat destination rule 400 disable
delete nat destination rule 401 disable
delete nat source rule 5010 disable
set service mdns repeater interface ‘eth1.11v11’
set service mdns repeater interface ‘eth1.13v13’
set service mdns repeater interface ‘eth1.131v131’
set service dhcp-relay interface ‘eth0.67’
set service dhcp-relay interface ‘eth1.11’
set service dhcp-relay interface ‘eth1.13’
set service dhcp-relay interface ‘eth1.131’
set service dhcp-relay interface ‘eth2.7’
set service dhcp-relay relay-options relay-agents-packets ‘discard’
set service dhcp-relay server ‘192.168.67.241’
set service dhcp-relay server ‘192.168.67.242’
set service dns dynamic interface eth0.167 service namecheap host-name ‘[REMOVED]’
set service dns dynamic interface eth0.167 service namecheap login ‘[REMOVED]’
set service dns dynamic interface eth0.167 service namecheap password ‘[REMOVED]’
set service dns dynamic interface eth0.167 service namecheap protocol ‘[REMOVED]’
set service dns dynamic interface eth0.167 service namecheap server ‘dynamicdns.park-your-domain.com
commit

exit
[edit]

/config/scripts/vrrp-trans-fail.sh

mario@vyos009# cat /config/scripts/vrrp-trans-fail.sh
#!/bin/vbash
source /opt/vyatta/etc/functions/script-template

configure
set nat destination rule 200 disable
set nat destination rule 201 disable
set nat destination rule 399 disable
set nat destination rule 400 disable
set nat destination rule 401 disable
set nat source rule 5010 disable
delete interfaces ethernet eth0 vif 167 address dhcp
set interfaces ethernet eth0 vif 167 address 192.168.167.167/24
delete service mdns
delete service dhcp-relay
delete service dns
commit

exit
[edit]

And the permissions of the scripts

mario@vyos009# ls -la /config/scripts
total 20
drwxrwsr-x 2 root vyattacfg 4096 Jul 21 10:22 .
drwxrwsr-x 9 root vyattacfg 4096 Jul 21 10:24 …
-rwxr-xr-x 1 root vyattacfg 486 Jul 21 10:22 vrrp-trans-fail.sh
-rwxr-xr-x 1 root vyattacfg 1417 Jul 21 10:22 vrrp-trans-master.sh
-rwxr-xr-x 1 root vyattacfg 230 Jul 20 11:23 vyos-postconfig-bootup.script
[edit]

call me a liar now… i changed nothing since that message… and rebooted several times more out of frustration…

just checked and what do i see in the log file… its working :face_with_raised_eyebrow: :face_with_hand_over_mouth: :roll_eyes:

Jul 21 10:58:55 vyos009 keepalived-fifo.py[19013]: Running the command: /config/scripts/vrrp-trans-fail.sh
Jul 21 10:58:56 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth2.53v53 for vrrp_instance cam
Jul 21 10:58:56 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth0.67v67 for vrrp_instance dmz
Jul 21 10:58:56 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth0.79v79 for vrrp_instance download
Jul 21 10:58:56 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth1.131v131 for vrrp_instance guest
Jul 21 10:58:56 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth1.11v11 for vrrp_instance iot
Jul 21 10:58:56 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth1.13v13 for vrrp_instance lan
Jul 21 10:58:56 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth2.7v7 for vrrp_instance mgmt
Jul 21 10:58:57 vyos009 Keepalived_vrrp[18998]: vmac: Success removing VMAC interface eth0.17v17 for vrrp_instance public
Jul 21 10:58:57 vyos009 Keepalived_vrrp[18998]: Stopped
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: Registering Kernel netlink reflector
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: Registering Kernel netlink command channel
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: Opening file ‘/etc/keepalived/keepalived.conf’.
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: Starting SNMP subagent
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: SECURITY VIOLATION - scripts are being executed but script_security not enabled.
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (cam): Success creating VMAC interface eth2.53v53
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (dmz): Success creating VMAC interface eth0.67v67
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (download): Success creating VMAC interface eth0.79v79
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (guest): Success creating VMAC interface eth1.131v131
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (iot): Success creating VMAC interface eth1.11v11
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (lan): Success creating VMAC interface eth1.13v13
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (mgmt): Success creating VMAC interface eth2.7v7
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (public): Success creating VMAC interface eth0.17v17
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: Registering gratuitous ARP shared channel
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (cam) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (dmz) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (download) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (guest) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (iot) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (lan) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (mgmt) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 Keepalived_vrrp[21156]: (public) Entering BACKUP STATE (init)
Jul 21 10:58:57 vyos009 keepalived-fifo.py[21160]: Loaded configuration: {‘vrrp_groups’: {‘cam’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}, ‘dmz’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}, ‘download’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}, ‘guest’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}, ‘iot’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}, ‘lan’: {‘STOP’: ‘/config/scripts/vrrp-trans-fail.sh’, ‘FAULT’: ‘/config/scripts/vrrp-trans-fail.sh’, ‘BACKUP’: ‘/config/scripts/vrrp-trans-fail.sh’, ‘MASTER’: ‘/config/scripts/vrrp-trans-master.sh’}, ‘mgmt’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}, ‘public’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}}, ‘sync_groups’: {‘sync’: {‘STOP’: None, ‘FAULT’: None, ‘BACKUP’: None, ‘MASTER’: None}}}
Jul 21 10:59:12 vyos009 Keepalived_vrrp[21156]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 21 10:59:27 vyos009 Keepalived_vrrp[21156]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 21 10:59:42 vyos009 Keepalived_vrrp[21156]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 21 10:59:57 vyos009 Keepalived_vrrp[21156]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 21 11:00:12 vyos009 Keepalived_vrrp[21156]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 21 11:00:27 vyos009 Keepalived_vrrp[21156]: Warning: Failed to connect to the agentx master agent ([NIL]):

Yea I was going to say, I can’t fault your config and very similar works on mine!

I use scripts like this to unshut a pppoe interface when the primary goes away.

Agree, but i was at a point of questioning my sanity hehe, its been bugging me all morning (few hours!).
It is the first day I have actually enabled the second router for this VRRP, got all the teething issues sorted out apart from this script.

Was just investigating, is VRRP transition scripts supposed to run when the router starts or re-starts? Mine is not, can see from log VRRP does indeed go to BACKUP state but it does not run the script, unlike when i manually restart via ‘restart vrrp’ after it restarts.

Did you notice this @tjh and if so, are you dealing with it in some fashion? Thanks!

I only have scripts on my backup router - the idea for me being that if the primary isn’t there, well, it isn’t there at all. So I don’t see the point of asking it to do stuff when it becomes master - I only ask the backup router to shutdown its interfaces when it becomes the backup vrrp again, i.e. the master has recovered from whatever terrible thing happened to it in the first place.

I don’t have any problems rebooting the backup router and it not “doing things” but then again, its default state is always to have its pppoe interface shutdown, it only opens it up when it becomes master. I’ve never rebooted it when it was master to see what happens.

Yeah mine will be similar but there is real possibilities for restart while the backup is master, which is why I am investigating this.
I am planning on swapping from Hyper-V hypervisor to XCP-ng on the primary to start with which will take the master router down for some time while i play at leasure. Have had a lengthy primary server outage previously while awaiting a new power supply.

For me, it is for production home use, but still, its an excellent learning opportunity, i should see if the posst config script may be what i am looking for here, to restart vrrp, which may do the trick.

Tho I still would assume myself that the initial start of the router should call the script as it is a state change, just unsure why it is not calling it.

It may well be there’s logic in Vyos to say something like “If we’re still starting up then don’t execute any VRRP scripts just yet as we’re still in a state of flux”

It’s always a problem with networking gear, the chicken and egg, if something comes up before something else and then the first thing flaps because the second thing hadn’t finished initilising etc.

I have no knowledge of the actual workings of Vyos in that regard though, so my comment is purely guesswork!

Your educated guess is actually the same as mine as to how it may work.
I will have a play with it a bit later to see if I can work something out and update here in case someone is searching down the track.

Cheers!

1 Like

Hmm, before I got too far, did a few more restarts with absolutely no config changes.
Apart from having to manually run ‘restart vrrp’ to invoke the script, it is being temperamental again.
It is running, the logs show as script executing (as in post 2) but the commands themselves do not appear to be executed when the script is running, only once in a blue moon do they actually execute.

May need some additional help afterall as its testing my sanity again…

Just to make sure, I am actually checking the config after every single time I call ‘restart vrrp’

mario@vyos009:~ restart vrrp mario@vyos009:~ show configuration commands | grep service
set service dhcp-relay interface ‘eth0.67’
set service dhcp-relay interface ‘eth1.11’
set service dhcp-relay interface ‘eth1.13’
set service dhcp-relay interface ‘eth1.131’
set service dhcp-relay interface ‘eth2.7’
set service dhcp-relay relay-options relay-agents-packets ‘discard’
set service dhcp-relay server ‘192.168.67.241’
set service dhcp-relay server ‘192.168.67.242’
set service dns dynamic interface eth0.167 service namecheap host-name ‘[REMOVED]’
set service dns dynamic interface eth0.167 service namecheap login ‘[REMOVED]’
set service dns dynamic interface eth0.167 service namecheap password ‘[REMOVED]’
set service dns dynamic interface eth0.167 service namecheap protocol ‘namecheap’
set service dns dynamic interface eth0.167 service namecheap server ‘dynamicdns.park-your-domain.com
set service mdns repeater interface ‘eth1.11v11’
set service mdns repeater interface ‘eth1.13v13’
set service mdns repeater interface ‘eth1.131v131’
set service ssh listen-address ‘192.168.7.254’
set service ssh port ‘22’

When it is successful and I get surprised only the 2 ssh lines return of course. Not sure why it is so inconsistent :frowning:

Cheers!

oooooooooooooookkkkkkkkk, so I believe I am having some sort of brain fart and its only just clicked.
VRRP process has some sort of method to detect no state change between restarts so it does not run from memory.

To test this, i rebooted the primary and then wondered why I did not lose my RDP session into one of the servers, had a session open to backup server and can see clearly it has become Master and script executed.

Boy o boy was I glad, is there a way to force the transition script to run as I please by restarting vrrp?
Mainly surrounding testing but more importantly during restart of router.

So I found this interesting phabricator https://phabricator.vyos.net/T1350
Now, how can I invoke the STOP state? and there does not appear to be a “reload” in 1.3 which would be excellent to reload config, say on reboot.

Cheers!

Slightly off topic - where would be a good starting point to learn about the VRRP set up and scripts needed to made this all work?

I would like to implement something on my home network “just for fun” but don’t know where to start.

Hey @phillipmcmahon sounds like we are in the same boat. It wasnt too bad with the existing documentation to get started and some firewall logs to work out I was missing one set of firewall rules.

I will export the relevant configuration parts for vrrp and share here, give me today and/or tomorrow tho

1 Like

Hey @phillipmcmahon here is the relevant bits for VRRP in my setup, hopefully it gives you info to get started.

First my VRRP related firewall groups, primary = .252, backup .254, i only showed one here, but you need this for each VRRP group, if you are applying a different firewall to each interface as I am.

I am using the same group with both/all the required addresses for simplicity otherwise it gets tedious…
Also this allows for much simpler updating between configs on different routers as its all in the same address group.

This config is to be entered on both routers, due to the unique setup with address groups, its copy/paste into both routers once you have the interfaces added into the groups correctly.

set firewall group address-group ag-vrrp-dmz address ‘192.168.67.252’
set firewall group address-group ag-vrrp-dmz address ‘192.168.67.254’

Same as above, per each interface dealing with VRRP

set firewall name dmz-firewall rule 10 action ‘accept’
set firewall name dmz-firewall rule 10 description ‘Allow VRRP’
set firewall name dmz-firewall rule 10 destination group address-group ‘ag-vrrp-dmz’
set firewall name dmz-firewall rule 10 protocol ‘vrrp’
set firewall name dmz-firewall rule 10 source group address-group ‘ag-vrrp-dmz’

set firewall name firewall-dmz rule 10 action ‘accept’
set firewall name firewall-dmz rule 10 description ‘Allow VRRP’
set firewall name firewall-dmz rule 10 destination group address-group ‘ag-vrrp-dmz’
set firewall name firewall-dmz rule 10 protocol ‘vrrp’
set firewall name firewall-dmz rule 10 source group address-group ‘ag-vrrp-dmz’

This is all that I needed as I was using my management network 192.168.7.0 for conntrack sync
This is not required to be repeated

set firewall name firewall-mgmt rule 650 action ‘accept’
set firewall name firewall-mgmt rule 650 description ‘Accept Conntrack Sync’
set firewall name firewall-mgmt rule 650 destination group address-group ‘ag-ct_sync’
set firewall name firewall-mgmt rule 650 destination group port-group ‘pg-ct_sync’
set firewall name firewall-mgmt rule 650 protocol ‘udp’
set firewall name firewall-mgmt rule 650 source group address-group ‘ag-vrrp-mgmt’

set firewall name firewall-mgmt rule 651 action ‘accept’
set firewall name firewall-mgmt rule 651 description ‘Allow IGMP for Conntrack Sync’
set firewall name firewall-mgmt rule 651 destination group address-group ‘ag-igmp’
set firewall name firewall-mgmt rule 651 protocol ‘igmp’
set firewall name firewall-mgmt rule 651 source group address-group ‘ag-vrrp-mgmt’

set firewall name mgmt-firewall rule 650 action ‘accept’
set firewall name mgmt-firewall rule 650 description ‘Accept Conntrack Sync’
set firewall name mgmt-firewall rule 650 destination group address-group ‘ag-ct_sync’
set firewall name mgmt-firewall rule 650 destination group port-group ‘pg-ct_sync’
set firewall name mgmt-firewall rule 650 protocol ‘udp’
set firewall name mgmt-firewall rule 650 source group address-group ‘ag-vrrp-mgmt’

The whole VRRP configuration, here you can see all of my vlans, the only interface I am not VRRP’ing is WAN as I am at home, planning to add a 4G backup connection soon but it will not be via VRRP.

set high-availability vrrp group cam advertise-interval ‘1’
set high-availability vrrp group cam description ‘Cam’
set high-availability vrrp group cam hello-source-address ‘192.168.53.254’
set high-availability vrrp group cam interface ‘eth2.53’
set high-availability vrrp group cam peer-address ‘192.168.53.252’
set high-availability vrrp group cam preempt-delay ‘3’
set high-availability vrrp group cam priority ‘250’
set high-availability vrrp group cam rfc3768-compatibility
set high-availability vrrp group cam virtual-address ‘192.168.53.253/24’
set high-availability vrrp group cam vrid ‘53’
set high-availability vrrp group dmz advertise-interval ‘1’
set high-availability vrrp group dmz description ‘DMZ’
set high-availability vrrp group dmz hello-source-address ‘192.168.67.254’
set high-availability vrrp group dmz interface ‘eth0.67’
set high-availability vrrp group dmz peer-address ‘192.168.67.252’
set high-availability vrrp group dmz preempt-delay ‘3’
set high-availability vrrp group dmz priority ‘250’
set high-availability vrrp group dmz rfc3768-compatibility
set high-availability vrrp group dmz virtual-address ‘192.168.67.253/24’
set high-availability vrrp group dmz vrid ‘67’
set high-availability vrrp group download advertise-interval ‘1’
set high-availability vrrp group download description ‘Download’
set high-availability vrrp group download hello-source-address ‘192.168.79.254’
set high-availability vrrp group download interface ‘eth0.79’
set high-availability vrrp group download peer-address ‘192.168.79.252’
set high-availability vrrp group download preempt-delay ‘3’
set high-availability vrrp group download priority ‘250’
set high-availability vrrp group download rfc3768-compatibility
set high-availability vrrp group download virtual-address ‘192.168.79.253/24’
set high-availability vrrp group download vrid ‘79’
set high-availability vrrp group guest advertise-interval ‘1’
set high-availability vrrp group guest description ‘Guest’
set high-availability vrrp group guest hello-source-address ‘192.168.131.254’
set high-availability vrrp group guest interface ‘eth1.131’
set high-availability vrrp group guest peer-address ‘192.168.131.252’
set high-availability vrrp group guest preempt-delay ‘3’
set high-availability vrrp group guest priority ‘250’
set high-availability vrrp group guest rfc3768-compatibility
set high-availability vrrp group guest virtual-address ‘192.168.131.253/24’
set high-availability vrrp group guest vrid ‘131’
set high-availability vrrp group iot advertise-interval ‘1’
set high-availability vrrp group iot description ‘IOT’
set high-availability vrrp group iot hello-source-address ‘192.168.11.254’
set high-availability vrrp group iot interface ‘eth1.11’
set high-availability vrrp group iot peer-address ‘192.168.11.252’
set high-availability vrrp group iot preempt-delay ‘3’
set high-availability vrrp group iot priority ‘250’
set high-availability vrrp group iot rfc3768-compatibility
set high-availability vrrp group iot virtual-address ‘192.168.11.253/24’
set high-availability vrrp group iot vrid ‘11’
set high-availability vrrp group lan advertise-interval ‘1’
set high-availability vrrp group lan description ‘LAN’
set high-availability vrrp group lan hello-source-address ‘192.168.13.254’
set high-availability vrrp group lan interface ‘eth1.13’
set high-availability vrrp group lan peer-address ‘192.168.13.252’
set high-availability vrrp group lan preempt-delay ‘3’
set high-availability vrrp group lan priority ‘250’
set high-availability vrrp group lan rfc3768-compatibility
set high-availability vrrp group lan virtual-address ‘192.168.13.253/24’
set high-availability vrrp group lan vrid ‘13’
set high-availability vrrp group mgmt advertise-interval ‘1’
set high-availability vrrp group mgmt description ‘Management’
set high-availability vrrp group mgmt hello-source-address ‘192.168.7.254’
set high-availability vrrp group mgmt interface ‘eth2.7’
set high-availability vrrp group mgmt peer-address ‘192.168.7.252’
set high-availability vrrp group mgmt preempt-delay ‘3’
set high-availability vrrp group mgmt priority ‘250’
set high-availability vrrp group mgmt rfc3768-compatibility
set high-availability vrrp group mgmt virtual-address ‘192.168.7.253/24’
set high-availability vrrp group mgmt vrid ‘7’
set high-availability vrrp group public advertise-interval ‘1’
set high-availability vrrp group public description ‘Public’
set high-availability vrrp group public hello-source-address ‘192.168.17.254’
set high-availability vrrp group public interface ‘eth0.17’
set high-availability vrrp group public peer-address ‘192.168.17.252’
set high-availability vrrp group public preempt-delay ‘3’
set high-availability vrrp group public priority ‘250’
set high-availability vrrp group public rfc3768-compatibility
set high-availability vrrp group public virtual-address ‘192.168.17.253/24’
set high-availability vrrp group public vrid ‘17’
set high-availability vrrp sync-group sync member ‘cam’
set high-availability vrrp sync-group sync member ‘guest’
set high-availability vrrp sync-group sync member ‘mgmt’
set high-availability vrrp sync-group sync member ‘lan’
set high-availability vrrp sync-group sync member ‘iot’
set high-availability vrrp sync-group sync member ‘public’
set high-availability vrrp sync-group sync member ‘dmz’
set high-availability vrrp sync-group sync member ‘download’
set high-availability vrrp sync-group sync transition-script backup ‘/config/scripts/vrrp-trans-fail.sh’
set high-availability vrrp sync-group sync transition-script fault ‘/config/scripts/vrrp-trans-fail.sh’
set high-availability vrrp sync-group sync transition-script master ‘/config/scripts/vrrp-trans-master.sh’
set high-availability vrrp sync-group sync transition-script stop ‘/config/scripts/vrrp-trans-fail.sh’

Conntrack sync stuffs

set service conntrack-sync accept-protocol ‘tcp,udp,icmp’
set service conntrack-sync disable-external-cache
set service conntrack-sync event-listen-queue-size ‘16’
set service conntrack-sync failover-mechanism vrrp sync-group ‘sync’
set service conntrack-sync interface eth2.7
set service conntrack-sync listen-address ‘192.168.7.254’
set service conntrack-sync mcast-group ‘224.0.0.50’
set service conntrack-sync sync-queue-size ‘16’

set system conntrack expect-table-size ‘2048’
set system conntrack hash-size ‘32768’
set system conntrack modules gre disable
set system conntrack modules nfs disable
set system conntrack modules pptp disable
set system conntrack modules sip disable
set system conntrack modules sqlnet disable
set system conntrack modules tftp disable
set system conntrack table-size ‘262144’

I have also knocked up a static powershell script for my own setup that replaces and removes all required things to copy the config from primary to backup.

Export config commands from first router

show configuration commands > setup cmds original.txt

Copy this to your windows machine and use script (edited for your own needs), it will spit out setup cmds for second router.txt in the same folder that the script and the original commands are.

For some reason the Bold parts have replaced comments starting with #

$workdir = $PSScriptRoot
$origfile = “$PSScriptRoot\setup cmds original.txt”
$newfile = “$PSScriptRoot\setup cmds for second router.txt”
$content = get-content -Path $origfile

function replace-things {
[cmdletBinding()]
param(
[parameter(Mandatory = $true)]
[string]$linefind,
[parameter(Mandatory = $true)]
[string]$linerepl
)

Pre replacement verification

$isexists = $content -match $linefind
if ($isexists -eq $linefind) {
write-host “SUCCESS located: [$linefind]” -ForegroundColor Green
}
else {
write-host “!ERROR! not located: [$linefind]” -ForegroundColor Red
}

Do the actual replace

$script:content = $content -replace $linefind, $linerepl

Post replacement verification

$ischanged = $content -match $linerepl
if ($ischanged -eq $linerepl) {
write-host “SUCCESS replaced: [$linerepl]” -ForegroundColor Green
}
else {
write-host “!ERROR! unable to replace” -ForegroundColor Red
}

write-output “# -------------------------------------------------”
}

function remove-things {
[cmdletBinding()]
param(
[parameter(Mandatory = $true)]
[string]$linetoremove
)

Pre replacement verification

$isexists = $content -match $linetoremove
if ($isexists -like $linetoremove) {
write-host “SUCCESS located: [$linetoremove]” -ForegroundColor Green
}
else {
write-host “!ERROR! not located: [$linetoremove]” -ForegroundColor Red
}

Do the actual removal

$script:content = $content -notmatch $linetoremove

Post replacement verification

$isexists = $content -match $linetoremove
if (-not ($isexists -like $linetoremove)) {
write-host “SUCCESS not located: [$linetoremove]” -ForegroundColor Green
}
else {
write-host “!ERROR! unable to remove: [$linetoremove]” -ForegroundColor Red
}

write-output “# -------------------------------------------------”
}

Replace general things

replace-things -linefind “set system host-name ‘vyos007’” -linerepl “set system host-name ‘vyos009’”
replace-things -linefind “set service ssh listen-address ‘192.168.7.252’” -linerepl “set service ssh listen-address ‘192.168.7.254’”
replace-things -linefind “set system ntp listen-address ‘192.168.7.252’” -linerepl “set system ntp listen-address ‘192.168.7.254’”
replace-things -linefind “set service conntrack-sync listen-address ‘192.168.7.252’” -linerepl “set service conntrack-sync listen-address ‘192.168.7.254’”

Replace VRRP related things

replace-things -linefind “set high-availability vrrp group cam hello-source-address ‘192.168.53.252’” -linerepl “set high-availability vrrp group cam hello-source-address ‘192.168.53.254’”
replace-things -linefind “set high-availability vrrp group cam peer-address ‘192.168.53.254’” -linerepl “set high-availability vrrp group cam peer-address ‘192.168.53.252’”
replace-things -linefind “set high-availability vrrp group dmz hello-source-address ‘192.168.67.252’” -linerepl “set high-availability vrrp group dmz hello-source-address ‘192.168.67.254’”
replace-things -linefind “set high-availability vrrp group dmz peer-address ‘192.168.67.254’” -linerepl “set high-availability vrrp group dmz peer-address ‘192.168.67.252’”
replace-things -linefind “set high-availability vrrp group download hello-source-address ‘192.168.79.252’” -linerepl “set high-availability vrrp group download hello-source-address ‘192.168.79.254’”
replace-things -linefind “set high-availability vrrp group download peer-address ‘192.168.79.254’” -linerepl “set high-availability vrrp group download peer-address ‘192.168.79.252’”
replace-things -linefind “set high-availability vrrp group guest hello-source-address ‘192.168.131.252’” -linerepl “set high-availability vrrp group guest hello-source-address ‘192.168.131.254’”
replace-things -linefind “set high-availability vrrp group guest peer-address ‘192.168.131.254’” -linerepl “set high-availability vrrp group guest peer-address ‘192.168.131.252’”
replace-things -linefind “set high-availability vrrp group iot hello-source-address ‘192.168.11.252’” -linerepl “set high-availability vrrp group iot hello-source-address ‘192.168.11.254’”
replace-things -linefind “set high-availability vrrp group iot peer-address ‘192.168.11.254’” -linerepl “set high-availability vrrp group iot peer-address ‘192.168.11.252’”
replace-things -linefind “set high-availability vrrp group lan hello-source-address ‘192.168.13.252’” -linerepl “set high-availability vrrp group lan hello-source-address ‘192.168.13.254’”
replace-things -linefind “set high-availability vrrp group lan peer-address ‘192.168.13.254’” -linerepl “set high-availability vrrp group lan peer-address ‘192.168.13.252’”
replace-things -linefind “set high-availability vrrp group mgmt hello-source-address ‘192.168.7.252’” -linerepl “set high-availability vrrp group mgmt hello-source-address ‘192.168.7.254’”
replace-things -linefind “set high-availability vrrp group mgmt peer-address ‘192.168.7.254’” -linerepl “set high-availability vrrp group mgmt peer-address ‘192.168.7.252’”
replace-things -linefind “set high-availability vrrp group public hello-source-address ‘192.168.17.252’” -linerepl “set high-availability vrrp group public hello-source-address ‘192.168.17.254’”
replace-things -linefind “set high-availability vrrp group public peer-address ‘192.168.17.254’” -linerepl “set high-availability vrrp group public peer-address ‘192.168.17.252’”

Lower VRRP priority for backup

replace-things -linefind “set high-availability vrrp group cam priority ‘255’” -linerepl “set high-availability vrrp group cam priority ‘250’”
replace-things -linefind “set high-availability vrrp group dmz priority ‘255’” -linerepl “set high-availability vrrp group dmz priority ‘250’”
replace-things -linefind “set high-availability vrrp group download priority ‘255’” -linerepl “set high-availability vrrp group download priority ‘250’”
replace-things -linefind “set high-availability vrrp group guest priority ‘255’” -linerepl “set high-availability vrrp group guest priority ‘250’”
replace-things -linefind “set high-availability vrrp group iot priority ‘255’” -linerepl “set high-availability vrrp group iot priority ‘250’”
replace-things -linefind “set high-availability vrrp group lan priority ‘255’” -linerepl “set high-availability vrrp group lan priority ‘250’”
replace-things -linefind “set high-availability vrrp group mgmt priority ‘255’” -linerepl “set high-availability vrrp group mgmt priority ‘250’”
replace-things -linefind “set high-availability vrrp group public priority ‘255’” -linerepl “set high-availability vrrp group public priority ‘250’”

Replace interface related things

replace-things -linefind “set interfaces ethernet eth0 vif 17 address ‘192.168.17.252/24’” -linerepl “set interfaces ethernet eth0 vif 17 address ‘192.168.17.254/24’”
replace-things -linefind “set interfaces ethernet eth0 vif 67 address ‘192.168.67.252/24’” -linerepl “set interfaces ethernet eth0 vif 67 address ‘192.168.67.254/24’”
replace-things -linefind “set interfaces ethernet eth0 vif 79 address ‘192.168.79.252/24’” -linerepl “set interfaces ethernet eth0 vif 79 address ‘192.168.79.254/24’”
replace-things -linefind “set interfaces ethernet eth1 vif 11 address ‘192.168.11.252/24’” -linerepl “set interfaces ethernet eth1 vif 11 address ‘192.168.11.254/24’”
replace-things -linefind “set interfaces ethernet eth1 vif 13 address ‘192.168.13.252/24’” -linerepl “set interfaces ethernet eth1 vif 13 address ‘192.168.13.254/24’”
replace-things -linefind “set interfaces ethernet eth1 vif 131 address ‘192.168.131.252/24’” -linerepl “set interfaces ethernet eth1 vif 131 address ‘192.168.131.254/24’”
replace-things -linefind “set interfaces ethernet eth2 vif 7 address ‘192.168.7.252/24’” -linerepl “set interfaces ethernet eth2 vif 7 address ‘192.168.7.254/24’”
replace-things -linefind “set interfaces ethernet eth2 vif 53 address ‘192.168.53.252/24’” -linerepl “set interfaces ethernet eth2 vif 53 address ‘192.168.53.254/24’”

Remove DHCP address from WAN interface

remove-things -linetoremove “set interfaces ethernet eth0 vif 167 address ‘dhcp’”

Remove hw-id from interface related things, add * at end of string instead of putting entire MAC address

remove-things -linetoremove “set interfaces ethernet eth0 hw-id*”
remove-things -linetoremove “set interfaces ethernet eth1 hw-id*”
remove-things -linetoremove “set interfaces ethernet eth2 hw-id*”

Write to new file

Out-File -FilePath $newfile -InputObject $content -Encoding utf8

Finally you can see my master and fail scripts for VRRP transition in my original post.

Its a lot of stuff but its mostly straight forward.
Cheers and good luck, ask about my config if not makes sense.

Top man - let me read through that and try to get my head round what it is doing.

You mentioned that there is no WAN VRRP, is the config you posted to provide LAN redundancy in that case?

Yes you are correct, it is so my entire complex network can continue to operate while I either play with the primary hypervisor or do updates etc while someone is watching tv through tvheadend in dmz vlan, so routed. Local media also in dmz or even internet streaming. Just means I don’t have to wait till late at night when everyone is in bed before I can do disruptive things.

Primary core services include vyos of course, domain controller, m$ DNS and dhcp, pihole and a few other things. The second hypervisor has all these services in high availability mode, through each service, not at VM replication level, so it’s seamless if a host goes down.

Would you be willing to share a little more about how your network is set up along with the hypervisor side of things.

Sounds like a great home project.

If you think best done via PM then feel free.

Phill

Happy to share.

My Hypervisors is currently Microsoft Hyper-V and I am running vyos on both of them with my 8 VLAN as detailed above, descriptions help why 8 but figure if im adding a couple, just go all out.
I have minimum of 2 NIC per hypervisor but plan to move to 10GBe now that my file server is separate box and 1GBe is a bit on the slow side when my hard drives are more then capable.

I have these additional VM’s on the primary hypervisor

  • M$ domain controller, serves AD, DNS, DHCP and NTP
  • Docker VM, serves currently only Ubiquity UniFi controller for my remaining AP’s and Portainer (stopped work here as planning to move to Kubernetes cluster)
  • Download box VM, contains its own VPN for outbound connections
  • Grafana VM, not highly used yet, got too much going on but will move it to Kubernetes cluster probably
  • OpenVPN community VM for inbound connections… still not switched to WireGuard, but may leave this yet as its on TCP/443, considering moving to a container
  • Opnsense VM for when I am testing if I want it…
  • PiHole VM for DNS filtering of ads, definitely moving to container
  • M$ Root Certificate Authority VM (Offline)
  • M$ Sub Certificate Authority VM
  • Random Server 2019 test box
  • Car workshop manuals VM
  • WSUS server, still unsure if want, chews ram like no tomorrow…
  • My BlueIris camera system runs on the OS of the primary hypervisor due to needing Intel graphics :confused:

I have these additional VM’s on the secondary hypervisor

  • M$ domain controller, serves AD, DNS, DHCP and NTP
  • OpenVPN community VM for inbound connections… this will get destroyed when I move to WireGuard
  • PiHole VM for DNS filtering of ads, definitely moving to container

Have an additional baremetal box for my file server, running OpenMediaVault on it and another docker instance where I am running 2x pci-e dual dvb tuners to tvheadend container for all over the air tv as well as running DeepStack AI container for the cameras.

My VyOS routers redirect any rogue DNS on port 53 to the piholes who communicate upwards with the M$ infrastructure.

All my services are sorted in some sort of system, most VM’s are dual homed with a DMZ and mgmt VLAN (domain controllers, piholes, CA services, WSUS etc) so I can allow port for RDP/SSH/admin interface to be used from the management vlan only accessible to permissible IP’s while the actual services on those boxes are accessible over dmz vlan.My hypervisors are also only dual homed as above, eg: my camera traffic is routed from the camera vlan

Download box runs over download vlan
OpenVPN runs over public vlan
Routers, WiFi contollers, switches run on management vlan
My cable modem is sitting in its own vlan

Cameras run on their own VLAN that has no in or out of internet, only about 2 or 3 rules for local traffic allowed to or from.

I think that is majority of the noteworthy things about my network/setup. Itching to get the Kubernetes cluster going for some reason.

1 Like