DMVPN - GRE routed clear text when IPSec down

Good observation- I haven’t tested point to point tunnels.

For DMVPN on VyOS 1.4, a quick fix in the template file /usr/share/vyos/templates/ipsec/swanctl/profile.j2 will stop all but one or two GRE packets

I think the full fix should be as simple as having a firewall chain / table that can catch the outbound traffic.

Hopefully someone with some actual nftables knowledge can weigh in here - this is outside my area.

I installed VyOS 1.4 and have tried to figure out how to use nftables to allow gre ipsec packets and block non ipsec protected gre but I can’t seem to figure it out. nftables logic does not seem to work like iptables.

OK, here is what I finally came up with to block clear text gre outbound using nftables on VyOS 1.4. This seems to work in my virtual lab. Normal VyOS firewall rules can be used to allow ipsec/gre inbound and block all other gre clear text inbound.

# Allow GRE outbound if attached to ipsec policy and block all other clear text GRE
nft add table ip custom
nft add chain ip custom output '{ type filter hook output priority -10 ; }'
nft add rule ip custom output ip protocol gre ipsec out reqid * counter accept
nft add rule ip custom output ip protocol gre counter drop

I am not sure if matching ipsec reqid with a wildcard is really proper but it seems to work (though it looks weird in nft list output). Another option is to match any reqid that is not 0 (i.e. anything above 0). E.g.

# Allow GRE outbound if attached to ipsec policy and block all other clear text GRE
nft add table ip custom
nft add chain ip custom output '{ type filter hook output priority -10 ; }'
nft add rule ip custom output ip protocol gre ipsec out reqid != 0 counter accept
nft add rule ip custom output ip protocol gre counter drop

This also seems to work in my virtual lab.

1 Like

Ok, so I’ve been messing around with VyOS some more, and I came across zone policy. From my testing, it looks like I can block traffic outbound from the firewall itself using zone policy. I have created a firewall that looks like this

name LOCAL-WAN {
    default-action "accept"
    rule 5 {
        action "drop"
        protocol "gre"
    }
}

and applied it to my WAN zone from traffic coming from my LOCAL zone. This does solve the problem of preventing cleartext gre outbound, but also has the unintended effect of preventing the IPSec tunnel from actually coming up. At least, I think that is what is happening. This is what I get when I do a tcpdump on the “WAN” bridge

07:11:41.740912 IP 192.168.7.101.isakmp > 192.168.7.100.isakmp: isakmp: phase 2/others I inf[E]
07:11:41.741697 IP 192.168.7.101.isakmp > 192.168.7.100.isakmp: isakmp: phase 1 I ident
07:11:41.742137 IP 192.168.7.100.isakmp > 192.168.7.101.isakmp: isakmp: phase 2/others R inf[E]
07:11:41.742965 IP 192.168.7.100.isakmp > 192.168.7.101.isakmp: isakmp: phase 1 R ident
07:11:41.745304 IP 192.168.7.101.isakmp > 192.168.7.100.isakmp: isakmp: phase 1 I ident
07:11:41.748848 IP 192.168.7.100.isakmp > 192.168.7.101.isakmp: isakmp: phase 1 R ident
07:11:41.751049 IP 192.168.7.101.isakmp > 192.168.7.100.isakmp: isakmp: phase 1 I ident[E]
07:11:41.751977 IP 192.168.7.100.isakmp > 192.168.7.101.isakmp: isakmp: phase 1 R ident[E]
07:11:41.754859 IP 192.168.7.101.isakmp > 192.168.7.100.isakmp: isakmp: phase 2/others I oakley-quick[E]
07:11:41.760351 IP 192.168.7.100.isakmp > 192.168.7.101.isakmp: isakmp: phase 2/others R oakley-quick[E]
07:11:41.763177 IP 192.168.7.101.isakmp > 192.168.7.100.isakmp: isakmp: phase 2/others I oakley-quick[E]

Then a second or two later

07:11:42.713122 IP 192.168.7.100.isakmp > 192.168.7.101.isakmp: isakmp: phase 2/others R inf[E]

192.168.7.100 is the hub while 192.168.7.101 is the spoke


I think I understand the first two and last lines of your nftables rules, but I don’t quite understand the third one. My understanding of IPSec is that the payload is completely encrypted, so the firewall shouldn’t be able to match a gre packet inside of an IPSec packet, no? I don’t understand the reqid *, reqid != 0, and counter bits at all. I briefly looked up reqid as it pertains to nftables and IPSec, but couldn’t find much information. Could you point me towards a resource that would explain it?

Is it possible to make a rule that includes the same parameters using the VyOS firewall?

I ask because I’d much prefer to have the rules as part of the VyOS config itself. That is partly for simpler administration and partly to minimize risk of accidentally not moving the startup script file during a migration or upgrade - especially since there’s no warning or failsafe if the file doesn’t run and apply the firewall “patch”)

Thank you for your help!

See this link for info on how iptables/nftables works with ipsec. Basically, meta data is attached to packets as it is processed and this includes ipsec data for packets that are to be encrypted by ipsec. This ipsec meta data is attached to the packet before it is actually encrypted, and it is this meta data you match on. So in my example I am matching a gre packet that also has ipsec policy attached (based on reqid) and then I drop all other gre without ipsec meta data. The linked article also mentions that you can match based on peer src/dst IP but I found this does not work, I am guessing this is because with DMVPN I am using ipsec transport mode (which is normal for DMVPN) so there wouldn’t be IPSec peer IP information, thus I used reqid to match.

Nice find on being able to use firewall zone policies to match outbound local traffic. Based on the above info above, rule 5 would need to be action “accept” and match would be “protocol gre” AND “ipsec match-ipsec” and then you would need a following rule that blocks all other gre. The problem with this is, it doesn’t work in nftables for output hooks (it works for input though). When using “ipsec match-ipsec” VyOS is trying to configure the nftables rule with “meta ipsec exists” which, for whatever reason, is not allowed on output. You have to use the ipsec syntax I mentioned in my last post.

Not being able to match ipsec traffic on outbound traffic is unfortunate. There’s some kind of firewall rework going on - I’ll ask if this is something that can be fixed as a part of that.


To anyone coming across this with the same problem, you should know that p252’s nftables rules need to go in /config/scripts/vyos-preconfig-bootup.script. This is different from vyos-postconfig-bootup.script, which is created automatically. If you put the rules in the postconfig file, NHRP will start before the script is run and the firewall rules are applied. This will result in at least your NHRP registration request packet being leaked in the cleartext, which contains your cisco-authentication password.

Putting the rules in the preconfig script applies the rules before anything else starts, and successfully prevents any cleartext GRE from escaping once services start coming up. Also, you need to escape the * with a \ when putting the rules into a file - otherwise it will try to insert file names from the working directory into the command.