Error while start conntrackd
Hi folks
I am trying to enable conntrack-sync between a pair of VyOS routers - but running into an error with conntrackd not starting.
Here is the config on the active router:
service {
conntrack-sync {
accept-protocol tcp
accept-protocol udp
accept-protocol icmp
failover-mechanism {
vrrp {
sync-group sync1
}
}
ignore-address 127.0.0.1
ignore-address 192.168.10.111
interface eth1 {
peer 192.168.10.101
port 667
}
sync-queue-size 1
}
snmp {
community routers {
authorization ro
}
}
}
and the config on the standby router:
service {
conntrack-sync {
accept-protocol tcp
accept-protocol udp
accept-protocol icmp
failover-mechanism {
vrrp {
sync-group sync1
}
}
ignore-address 127.0.0.1
ignore-address 192.168.10.101
interface eth1 {
peer 192.168.10.111
port 667
}
sync-queue-size 1
}
snmp {
community routers {
authorization ro
}
}
}
But conntrackd fails to start on both the routers:
Jan 11 23:30:36 systemd[1]: Stopped Conntrack Daemon.
Jan 11 23:30:36 systemd[1]: Starting Conntrack Daemon...
Jan 11 23:30:36 conntrackd[97062]: [Thu Jan 11 23:30:36 2024] (pid=97062) [notice] reliable ctnetlink event delivery is ENABLED.
Jan 11 23:30:36 conntrackd[97062]: [Thu Jan 11 23:30:36 2024] (pid=97062) [notice] using user-space event filtering
Jan 11 23:30:36 conntrackd[97062]: [Thu Jan 11 23:30:36 2024] (pid=97062) [notice] netlink event socket buffer size has been set to 2097152 bytes
Jan 11 23:30:36 conntrackd[97062]: [Thu Jan 11 23:30:36 2024] (pid=97062) [notice] configuring helper `tns' with queuenum=5 and queuelen=0
Jan 11 23:30:36 conntrackd[97062]: [Thu Jan 11 23:30:36 2024] (pid=97062) [notice] policy name=tns expect_timeout=300 expect_max=1
Jan 11 23:30:36 conntrackd[97062]: [Thu Jan 11 23:30:36 2024] (pid=97062) [ERROR] trying to configure cthelper `tns': Invalid argument
Jan 11 23:30:36 conntrackd[97062]: [Thu Jan 11 23:30:36 2024] (pid=97062) [ERROR] conntrackd cannot start, please review your configuration
Jan 11 23:30:36 conntrack-tools[97062]: reliable ctnetlink event delivery is ENABLED.
Jan 11 23:30:36 conntrack-tools[97062]: using user-space event filtering
Jan 11 23:30:36 conntrack-tools[97062]: netlink event socket buffer size has been set to 2097152 bytes
Jan 11 23:30:36 conntrack-tools[97062]: configuring helper `tns' with queuenum=5 and queuelen=0
Jan 11 23:30:36 conntrack-tools[97062]: policy name=tns expect_timeout=300 expect_max=1
Jan 11 23:30:36 conntrack-tools[97062]: trying to configure cthelper `tns': Invalid argument
Jan 11 23:30:36 systemd[1]: conntrackd.service: Main process exited, code=exited, status=1/FAILURE
Jan 11 23:30:36 conntrack-tools[97062]: conntrackd cannot start, please review your configuration
Jan 11 23:30:36 systemd[1]: conntrackd.service: Failed with result 'exit-code'.
Jan 11 23:30:36 systemd[1]: Failed to start Conntrack Daemon.
Jan 11 23:30:36 systemd[1]: conntrackd.service: Service RestartSec=100ms expired, scheduling restart.
Jan 11 23:30:36 systemd[1]: conntrackd.service: Scheduled restart job, restart counter is at 5.
Jan 11 23:30:36 systemd[1]: Stopped Conntrack Daemon.
Jan 11 23:30:36 systemd[1]: conntrackd.service: Start request repeated too quickly.
Jan 11 23:30:36 systemd[1]: conntrackd.service: Failed with result 'exit-code'.
Jan 11 23:30:36 systemd[1]: Failed to start Conntrack Daemon.
Could I get some help? Why is the conntrack helper saying invalid argument?
Thanks in advance!
tjh
January 12, 2024, 1:10am
2
To help narrow down the problem, please mention what version of Vyos you’re using.
I am using VyOS 1.3x (Equulus).
Satish
marc_s
January 12, 2024, 7:09am
5
Please post output of show version.
Thanks folks - here is the output
root@vyos:/# show version
Version: VyOS 1.3-rolling-202308160027
Release train: equuleus
Built by: vyos_bld@0af5c065fb7b
Built on: Wed 16 Aug 2023 00:27 UTC
Build UUID: 7df8c206-1b6e-42d7-be35-e7a68116f5f7
Build commit ID: cba0c9e4bdcca9-dirty
Architecture: x86_64
Boot via: installed image
System type: VMware guest
Hardware vendor: VMware, Inc.
Hardware model: VMware Virtual Platform
Hardware S/N: VMware-56 4d 7e ea 05 63 93 1d-f2 3e 52 93 a1 a3 9d 3f
Hardware UUID: ea7e4d56-6305-1d93-f23e-5293a1a39d3f
Copyright: VyOS maintainers and contributors
root@vyos:/#
Hi folks
Any suggestions or ideas?
Thanks in advance!
marc_s
January 15, 2024, 3:57pm
8
Looks a lot like ⚓ T4417 VRRP doesn't start with conntrack-sync but I’m not sure. One of the maintainers should be able to tell you if it’s the same and if equuleus is affected.
Thanks @marc_s .
@Viacheslav - could you please help confirm? The error message in my case is different though - its erroring on trying configure cthelper tns.
Satish
If you know how to reproduce the bug open please a bug report on the https://vyos.dev/
You can also check 1.4-RC1 VyOS 1.4.0-rc1 release candidate
@Viacheslav So looks like upgrading to VyOS 1.4-RC1 fixes the issue. Here is the same config which works in VyOS 1.4-RC1 but not in 1.3-Equuleus.
high-availability {
vrrp {
group 72 {
advertise-interval 1
interface eth1
no-preempt
priority 253
track {
interface eth0
}
transition-script {
backup "/etc/airgap/vyos/vrrp/vrrp_notify_script.sh BACKUP"
fault "/etc/airgap/vyos/vrrp/vrrp_notify_script.sh FAULT"
master "/etc/airgap/vyos/vrrp/vrrp_notify_script.sh MASTER"
}
virtual-address 10.160.160.1/24 {
interface eth0.1600
}
virtual-address 10.170.170.1/24 {
interface eth0.1700
}
virtual-address 192.168.10.200/32 {
}
vrid 72
}
sync-group sync1 {
member 72
}
}
}
service {
conntrack-sync {
accept-protocol tcp
accept-protocol udp
accept-protocol icmp
failover-mechanism {
vrrp {
sync-group sync1
}
}
ignore-address 127.0.0.1
ignore-address 192.168.10.111
interface eth1 {
peer 192.168.10.101
port 667
}
sync-queue-size 1
}
snmp {
community routers {
authorization ro
}
}
}
Now I don’t see the error anymore:
vyos@fixdevtest--DEVTESTC--DEVTESTC-1# run show log conntrack-sync
Jan 16 02:28:21 systemd[1]: Starting conntrackd.service - Conntrack Daemon...
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] reliable ctnetlink event delivery is ENABLED.
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] using user-space event filtering
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] netlink event socket buffer size has been set to 2097152 bytes
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] configuring helper `tns' with queuenum=5 and queuelen=0
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] policy name=tns expect_timeout=300 expect_max=1
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] helper `tns' configured successfully
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] configuring helper `rpc' with queuenum=4 and queuelen=0
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] policy name=rpc expect_timeout=300 expect_max=1
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] helper `rpc' configured successfully
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] configuring helper `rpc' with queuenum=3 and queuelen=0
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] policy name=rpc expect_timeout=300 expect_max=1
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] helper `rpc' configured successfully
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] initialization completed
Jan 16 02:28:21 conntrackd[15578]: [Tue Jan 16 02:28:21 2024] (pid=15578) [notice] -- starting in console mode --
Jan 16 02:28:21 conntrack-tools[15578]: reliable ctnetlink event delivery is ENABLED.
Jan 16 02:28:21 systemd[1]: Started conntrackd.service - Conntrack Daemon.
Jan 16 02:28:21 conntrack-tools[15578]: using user-space event filtering
Jan 16 02:28:21 conntrack-tools[15578]: netlink event socket buffer size has been set to 2097152 bytes
Jan 16 02:28:21 conntrack-tools[15578]: configuring helper `tns' with queuenum=5 and queuelen=0
Jan 16 02:28:21 conntrack-tools[15578]: policy name=tns expect_timeout=300 expect_max=1
Jan 16 02:28:21 conntrack-tools[15578]: helper `tns' configured successfully
Jan 16 02:28:21 conntrack-tools[15578]: configuring helper `rpc' with queuenum=4 and queuelen=0
Jan 16 02:28:21 conntrack-tools[15578]: policy name=rpc expect_timeout=300 expect_max=1
Jan 16 02:28:21 conntrack-tools[15578]: helper `rpc' configured successfully
Jan 16 02:28:21 conntrack-tools[15578]: configuring helper `rpc' with queuenum=3 and queuelen=0
Jan 16 02:28:21 conntrack-tools[15578]: policy name=rpc expect_timeout=300 expect_max=1
Jan 16 02:28:21 conntrack-tools[15578]: helper `rpc' configured successfully
Jan 16 02:28:21 conntrack-tools[15578]: initialization completed
Jan 16 02:28:21 conntrack-tools[15578]: -- starting in console mode --
also:
vyos@fixdevtest--DEVTESTC--DEVTESTC-1# run show conntrack table ipv4
sudo: unable to resolve host fixdevtest--DEVTESTC--DEVTESTC-1: System error
Id Original src Original dst Reply src Reply dst Protocol State Timeout Mark Zone
---------- -------------------- -------------------- -------------------- -------------------- ---------- ----------- --------- ------ ------
2797110693 192.168.10.111:41036 34.82.150.81:17885 34.82.150.81:17885 192.168.10.111:41036 tcp ESTABLISHED 420815 0
3749478988 192.168.10.111:45742 34.82.150.81:17885 34.82.150.81:17885 192.168.10.111:45742 tcp ESTABLISHED 420813 0
3143078564 192.168.10.111:53267 50.18.131.86:443 50.18.131.86:443 192.168.10.111:53267 tcp ESTABLISHED 420812 0
1062631985 192.168.10.102:49840 192.168.10.111:22 192.168.10.111:22 192.168.10.102:49840 tcp ESTABLISHED 420813 0
513233446 127.0.0.1:51558 127.0.0.1:33045 127.0.0.1:33045 127.0.0.1:51558 tcp ESTABLISHED 420804 0
3563657564 192.168.10.111:41038 34.82.150.81:17885 34.82.150.81:17885 192.168.10.111:41038 tcp ESTABLISHED 420816 0
3079571785 192.168.10.111:41598 34.82.150.81:17885 34.82.150.81:17885 192.168.10.111:41598 tcp ESTABLISHED 420816 0
1759797412 192.168.10.111:53194 35.233.163.243:17885 35.233.163.243:17885 192.168.10.111:53194 tcp ESTABLISHED 420798 0
2331907681 192.168.10.111:41608 34.82.150.81:17885 34.82.150.81:17885 192.168.10.111:41608 tcp ESTABLISHED 420774 0
3624957823 192.168.10.111:40460 34.105.92.200:17885 34.105.92.200:17885 192.168.10.111:40460 tcp ESTABLISHED 420793 0
212974829 127.0.0.1:46208 127.0.0.1:3306 127.0.0.1:3306 127.0.0.1:46208 tcp ESTABLISHED 420713 0
1960861238 192.168.10.1:5173 192.168.10.200:179 192.168.10.200:179 192.168.10.1:5173 tcp ESTABLISHED 420799 0
[edit]
vyos@fixdevtest--DEVTESTC--DEVTESTC-1# run show conntr
conntrack conntrack-sync
[edit]
vyos@fixdevtest--DEVTESTC--DEVTESTC-1# run show conntrack-sync statistics
sudo: unable to resolve host fixdevtest--DEVTESTC--DEVTESTC-1: System error
cache internal:
current active connections: 12
connections created: 12 failed: 0
connections updated: 0 failed: 0
connections destroyed: 0 failed: 0
cache external:
current active connections: 0
connections created: 0 failed: 0
connections updated: 0 failed: 0
connections destroyed: 0 failed: 0
traffic processed:
0 Bytes 0 Pckts
UDP traffic (active device=eth1):
1616 Bytes sent 0 Bytes recv
202 Pckts sent 0 Pckts recv
0 Error send 0 Error recv
message tracking:
0 Malformed msgs 0 Lost msgs
Main Table Statistics:
[edit]
But upgrading to VyOS 1.4 means we need to revalidate all the CLI configuration we are using?
@Viacheslav Do you know when VyOS 1.4 LTS would be released? Any rough timeframe, as we are trying to evaluate if we should move to that.
Thanks!
Read our blog posts, there will be no other sources of information of releases.
1 Like