Hairpin NAT, auto-populating address groups and dynamic interfaces

Development
1

There’s a few tickets open relating to issues with hairpinning on dynamic interfaces, where the rule needs to know the interface address and it cannot be statically assigned, including:

Often these problems have been fixed via split horizon DNS (internal DNS resolves to internal IP, avoiding the NAT), something like EdgeOS’ ADDRvX_iface automatic address groups or user-scripted rough equivalents.

I’d like to have a crack at an integrated solution since it’s a common and annoying problem, and I’ve got a couple of ideas on how best to present it in the config.

Under the hood, the foundation piece will likely be a set of dhclient/pppd/if-up/down.d hooks that populate an nft set based on IP config and changes.

1. Allow a dynamic-group to be populated from any interface with explicit config

Something along the lines of:

set firewall group dynamic-group address-group WAN-EXT 
set interface ethernet eth0 address dhcp
set interface ethernet eth0 tracking-address-group WAN-EXT

Each DA_WAN-EXT set would be populated appropriately as IPs are assigned and changed.

I’ve targeted dynamic-groups as we already know they’ll be changing at runtime, there won’t be any assumptions made about config matching runtime state. I haven’t yet checked out whether they’re subject to unexpected flushing or other potential problems that would break this idea.

The user gets to control exactly how the groups are named and used, multiple interfaces can feed a single set (hopefully - I still need to check that we can track state in all the required hooks for adds/removes) and backs into the dynamic-group work already done.

2. Create a tracking-group type that works in a similar fashion

Very similar to the above, configured as:

set firewall group tracking-group WAN-EXT interface eth0
set firewall group tracking-group WAN-EXT interface pppoe1
set interface ethernet eth0 address dhcp

Working on a similar schema, but not exposed at all for additional explicit configuration.

3. Add match syntax in the firewall/NAT/etc rules, to match interface addresses:

In this scenario, we automatically create internal sets along the lines of IA_eth0 for any configured interface and always populate them from hooks.

Then, configuring at the match side, we always have access to it as long as the interfaces exist:

set interface ethernet eth0 address dhcp
set nat destination rule 100 destination interface-address eth0
set firewall ipv4 output filter rule 100 source interface-address pppoe1 [...]

This won’t need to worry about mixing together multiple interface states into a single set and makes it explicit what we’re doing - there’s no magical virtual address-group.

4. Automatic address-groups with defined names, similar to EdgeOS

The internal implementation of this is similar to extending the match syntax, but we move it into firewall group handling.

It’ll be immediately familiar to people coming from EdgeOS and is likely the simplest option to implement.

To avoid any namespace clashes with existing configs, especially when people have munged an old EdgeOS config into VyOS, it would ideally be defined as a new type of tracking-group, similar to #2 above.

Anyone have any opinions or suggestions?

2

Are there any places where this could be useful other than the firewall/NAT?

For example

set resource-group local-addresses <foo> interface eth0

or like this, and re-use the “resource”

set system resource-group <foo> interface all
set service ssh listen-address local-addresses <foo>
3

I hadn’t considered extending it beyond nftables, not a bad idea, but a hell of a lot more work :). I’d need a runtime state cache synchronising all the pieces and keeping track of what requires notification.

There’s a few places where it could conceivably be useful:

  • Service bindings, like your example, may not be super useful for dynamics. Changing socket bindings on a daemon usually involves a restart. Statics however, could be handy, especially for automation or config replication.
  • Feeding dynamic data into route policy elements. Dynamic routing protocols are already capable of picking up dynamic addressing without managed elements fed to prefix-lists or route-maps, but someone might have a use case.
  • qos policy elements are more useful within some narrow use-cases, like speed-limiting a hairpin NAT. For most cases you’d just attach an appropriate policy straight to the interface and not worry about the addressing.

I wouldn’t want to jump on this level of change right out of the gate, this suggestion is my first attempt at working on multiple pieces of VyOS at the same time. It’s not a bad idea to make the underlying implementation generic or hookable so that other stuff can be added easily later. If there’s use cases, it can just be extended.

EDIT: I found the ticket I was trying to find when originally replying: ⚓ T5647 Extend failover route functionality to use dynamically assigned interface next hops

Started getting some crazy ideas about generically tracking a lot of dynamically determined things but I didn’t want to get that carried away on my first little project :expressionless:.

4

The means to solve this pretty much already exist, so I don’t think you really need to worry about any kind of tracking. You can update the dynamic set with the input chain in nft. Something like this works:

set firewall group dynamic-group address-group Dyn_Test

set firewall ipv4 input filter rule 3 action 'continue'
set firewall ipv4 input filter rule 3 add-address-to-group destination-address address-group 'Dyn_Test'
set firewall ipv4 input filter rule 3 inbound-interface name 'eth0'
set firewall ipv4 input filter rule 3 state 'established'
set firewall ipv4 input filter rule 3 state 'related'

l0crian@NPB7# sudo nft -a list set vyos_filter DA_Dyn_Test
table ip vyos_filter {
        set DA_Dyn_Test { # handle 8
                type ipv4_addr
                size 65535
                flags dynamic,timeout
                elements = { <my public IP> timeout 15s expires 14s792ms }
        }
}

You would need to allow the dynamic groups to be called as source/destination under nat. And ideally the ability to limit dynamic sets to a given size would be useful, so you could just set it to 1.

5

A lot of the infrastructure is already there with the dynamic group implementation, but needing to wait for some inbound traffic to populate a group so internal traffic hairpinning works - it’s a bit odd. It’d be cleaner if they directly populate the moment an interface changes state.

Practically, on the internet with a public IP you’ll get hammered immediately, so the workaround is great for most instances of this problem.

6

DHCP ack would populate the set immediately. There’d be no difference in timing between either solution.

7

Yep that’s true.

What about PPPoE or an OpenVPN client?

8

Not sure why you’d ever map a service to a non-static OpenVPN interface’s IP. With PPPoE it’ll depend on how the IP is issued obviously.

To be clear, I’m not saying there’s any issue with your plan, and my input is entirely unimportant to whether the VyOS team says “go forth”. Just providing my input.

The way I look at this is no matter what, the users will need to know how to get to the new IP, which would likely be through some DDNS service, which would generate return traffic to be able to populate that set. That need also makes any change of the interface IP non-immediate from a user perspective.

If the DDNS service is not on the internet (e.g. tied to dnsmasq, bind, etc…), then there’s no reason for hairpin NAT, since split DNS should be used.

You’re identifying problems and working to solve them, which is awesome! So I won’t be upset either way with how it’s solved. Just providing my input on exploring if it can (or should) be done with existing mechanisms before adding new ones.

2 Likes
9

No I get it and appreciate the input. There are workarounds that don’t involve chasing the external interface IP at all - in most cases I’d assume it’s discovered from DNS, an internal DNS override to the internal IP means there’s no problem. I mention this in my original post.

However, I saw there were a few people going above and beyond with scripted command injection to make something work, some forum discussion and a couple of open tickets, it seemed like a good opportunity for a first non-trivial contribution to VyOS, to learn modifying the config schema and extending nftables at least.

In my experience, if there’s a weird unexpected way to do something, someone will find it and want it to work like everything else. Thus, I’m considering any dynamically assigned interface I can think of. I threw the question here in the forum to see if there were any other things I hadn’t yet thought of.

3 Likes
10

What you propose would certainly be easier for users to grasp. :grinning: