Hi Nicolas,
Thanks for taking a look as this is a weird one. I have spent too many hours trying to work out why it doesn’t work.
Here is an example I took earlier today whilst testing.
I started off with the rule
set firewall ipv4 name LAN-WAN rule 265 action 'accept'
set firewall ipv4 name LAN-WAN rule 265 description 'Allow WEB JAKOB LAN-WAN'
set firewall ipv4 name LAN-WAN rule 265 destination group port-group 'PG_WEB'
set firewall ipv4 name LAN-WAN rule 265 log
set firewall ipv4 name LAN-WAN rule 265 protocol 'tcp_udp'
set firewall ipv4 name LAN-WAN rule 265 source group address-group 'AG_LAN_TIMEGROUP'
set firewall ipv4 name LAN-WAN rule 265 state 'new'
Check the logs - all is as expected.
Feb 02 14:39:32 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=104.21.83.41 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=41036 DF PROTO=TCP SPT=55718 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:39:32 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=104.237.62.212 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=22367 DF PROTO=TCP SPT=39670 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:39:32 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=104.237.62.212 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=16778 DF PROTO=TCP SPT=39678 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:39:32 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=104.21.83.41 LEN=1278 TOS=0x00 PREC=0x00 TTL=63 ID=31994 DF PROTO=UDP SPT=43132 DPT=443 LEN=1258
Feb 02 14:39:33 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=172.64.149.23 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=38804 DF PROTO=TCP SPT=44838 DPT=80 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:39:33 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=172.64.149.23 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=27416 DF PROTO=TCP SPT=44852 DPT=80 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:39:33 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=103.101.129.194 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=35640 DF PROTO=TCP SPT=51580 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:40:30 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.251.221.74 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=56283 DF PROTO=TCP SPT=50308 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
I then add the rules which should theoretically work without blocking anything, as it’s 2:40PM
set firewall ipv4 name LAN-WAN rule 265 time starttime 06:30:00
set firewall ipv4 name LAN-WAN rule 265 time stoptime 22:30:00
Straight away it starts blocking the traffic?
Feb 02 14:48:07 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=60767 DF PROTO=TCP SPT=39980 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:48:07 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=20.43.109.14 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=1250 DF PROTO=TCP SPT=44460 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:48:07 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=46502 DF PROTO=TCP SPT=39992 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:48:07 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=20.43.109.14 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=14305 DF PROTO=TCP SPT=44462 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:48:07 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=57041 DF PROTO=TCP SPT=39994 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
I then go and shrink the time to 2:45PM till 3:00PM and it starts to work again.
set firewall ipv4 name LAN-WAN rule 265 time starttime 14:52:00
set firewall ipv4 name LAN-WAN rule 265 time starttime 15:00:00
The logs show it is working as expected. I have no idea why it is doing this?
Feb 02 14:51:56 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=104.20.151.16 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=38627 DF PROTO=TCP SPT=41922 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:51:57 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=25179 DF PROTO=TCP SPT=57264 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:51:58 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=25180 DF PROTO=TCP SPT=57264 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:52:00 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=270 DF PROTO=TCP SPT=40374 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:52:00 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=25181 DF PROTO=TCP SPT=57264 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:52:00 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=152.195.38.76 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=46366 DF PROTO=TCP SPT=51972 DPT=80 WINDOW=64240 R
...
....
.....
Feb 02 14:58:02 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=16504 DF PROTO=TCP SPT=40484 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:58:02 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=54384 DF PROTO=TCP SPT=40496 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:58:13 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=63549 DF PROTO=TCP SPT=54532 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:58:27 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.250.70.142 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=32339 DF PROTO=TCP SPT=36176 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 14:58:30 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=37.0.81.241 LEN=1278 TOS=0x00 PREC=0x00 TTL=63 ID=40766 DF PROTO=UDP SPT=41871 DPT=443 LEN=1258
Feb 02 14:59:34 box kernel: [ipv4-NAM-LAN-WAN-265-A]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.250.70.142 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=57166 DF PROTO=TCP SPT=48956 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 15:00:05 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.250.70.226 LEN=1278 TOS=0x00 PREC=0x00 TTL=63 ID=30413 DF PROTO=UDP SPT=41598 DPT=443 LEN=1258
Feb 02 15:00:05 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.250.70.226 LEN=1278 TOS=0x00 PREC=0x00 TTL=63 ID=30414 DF PROTO=UDP SPT=41598 DPT=443 LEN=1258
Feb 02 15:00:05 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.250.70.226 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=43394 DF PROTO=TCP SPT=48010 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Feb 02 15:00:05 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.250.70.226 LEN=1278 TOS=0x00 PREC=0x00 TTL=63 ID=30415 DF PROTO=UDP SPT=41598 DPT=443 LEN=1258
Feb 02 15:00:06 box kernel: [ipv4-LAN-WAN-default-D]IN=eth1 OUT=eth0 MAC=2c:f3:5d:d6:b3:8b:8c:8d:28:e5:44:b7:08:00 SRC=192.168.100.62 DST=142.250.70.226 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=35012 DF PROTO=TCP SPT=48020 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
I can see the configuration is there when I run sudo nft list ruleset
There are no other rules blocking it.
chain NAME_LAN-WAN {
....
....
ct state new meta l4proto { tcp, udp } th dport @P_PG_WEB ip saddr @A_AG_LAN_TIMEGROUP meta hour >= "06:30" meta hour < "22:30" log prefix "[ipv4-NAM-LAN-WAN-265-A]" counter packets 0 bytes 0 accept comment "ipv4-NAM-LAN-WAN-265"
...
}
Version: VyOS 1.4.0-rc3
Release train: sagitta
Built by: Sentrium S.L.
Built on: Thu 18 Jan 2024 19:21 UTC
Build UUID: 8603fe3e-29bd-4669-9427-cc07110bd272
Build commit ID: 651ff15892ade4
Architecture: x86_64
Boot via: installed image
System type: bare metal
Hardware vendor: OEM
Hardware model: Default string
Hardware S/N: 1234567890
Hardware UUID: 03000200-0400-0500-0006-000700080009
Copyright: VyOS maintainers and contributors
: A29FC87B (time.cloudflare.com)
Stratum : 4
Ref time (UTC) : Fri Feb 02 10:54:23 2024
System time : 0.000059460 seconds fast of NTP time
Last offset : +0.000190837 seconds
RMS offset : 0.000416427 seconds
Frequency : 20.839 ppm fast
Residual freq : +0.005 ppm
Skew : 0.314 ppm
Root delay : 0.010854744 seconds
Root dispersion : 0.002697953 seconds
Update interval : 1027.1 seconds
Leap status : Normal
I tried using UTC time but that doesn’t work even for the compressed time test.
Above my 2PM is 3AM UTC and that just did not allow traffic full stop.
So I am using zone based firewall rules with global policy rules
set firewall global-options state-policy established action 'accept'
set firewall global-options state-policy invalid action 'drop'
set firewall global-options state-policy invalid log
set firewall global-options state-policy related action 'accept'
There is nothing special in my configuration… all pretty stock standard.
I’ve run out of places where to look for troubleshooting this.
Kind Regards