Running RPKI on different VRF

[hanisirfan]

I’ve created a separate “mgmt“ VRF for out-of-band management and added the default route for it with this command.

set protocols static route 0.0.0.0/0 interface eth9 vrf 'mgmt'
set vrf name mgmt protocols static route 0.0.0.0/0 next-hop 10.0.0.97

vyos@router1~$ nc -v -z -w 3 10.0.2.98 3323
nc: connect to 10.0.2.98 port 3323 (tcp) timed out: Operation now in progress
vyos@router1:~$ force vrf mgmt
vyos@router1:~$ nc -v -z -w 3 10.0.2.98 3323
Connection to 10.0.2.98 3323 port [tcp/*] succeeded!

Netcat works from mgmt VRF as expected but doesn’t work from default VRF. I assume it’s not a routing issue in the OOB network since my firewall (pfSense) does receive the packets, but with RST packet. Netcat from different servers in the same RPKI server LAN passed. Other devices in the OOB network also works except VyOS routers. Any idea why?

[L0crian]

You have the route from default to mgmt, but did you add the return route from mgmt to default?

[hanisirfan]

Sorry but I don’t understand what you meant. Just FYI, I ran a Wireshark capture to troubleshoot this (I don’t know what else to do). When I run the netcat command from default VRF, I see RST flag on the TCP packet sourcing from VyOS (10.0.0.102).

Meanwhile, when running the netcat command from the mgmt VRF, it close correctly.

[L0crian]

Sorry but I don’t understand what you meant.

Let’s say you have the IP 10.0.0.102 inside your default VRF, and follow the default to 10.0.2.98 inside the mgmt VRF. When traffic returns to the mgmt VRF from 10.0.2.98, it needs to know how to get back to 10.0.0.102.

Your wireshark captures don’t quite make sense to me. Do you have the 10.0.0.0 subnet in both the default and mgmt VRFs?

[hanisirfan]

I hope you can understand the network based on this poor diagram lol

Subnets:
10.0.0.96/27 (OOB network in mgmt VRF)
10.0.2.96/27 (RPKI servers network)

Management interface > OOB switch > OOB firewall and RPKI servers > OOB firewall is all routed with no NAT.

TBH, I don’t have much experience in VRF as of now. This is part of my learning process.

[L0crian]

Yeah, that makes sense. You need to identify the address in the default VRF that is originating the connection to the RPKI servers. Then make sure the mgmt VRF knows how to get back to that.

A useful feature for this is veth interfaces. You can do something like this:

set interfaces virtual-ethernet veth1 address '10.0.199.1/30'
set interfaces virtual-ethernet veth1 peer-name 'veth2'

set interfaces virtual-ethernet veth2 address '10.0.199.2/30'
set interfaces virtual-ethernet veth2 peer-name 'veth1'
set interfaces virtual-ethernet veth2 vrf 'mgmt'

Then:

set protocols static route 0.0.0.0/0 next-hop 10.0.199.2

set vrf name mgmt protocols static route <the IP that originated the connection> next-hop 10.0.199.1

Or:

set protocols ospf interface veth1 area 0
set protocols ospf interface veth1 network point-to-point
set protocols ospf interface <interface that owns the IP that originated the connection> area 0
set protocols ospf interface <interface that owns the IP that originated the connection> passive

set vrf name mgmt protocols ospf interface veth2 area 0 
set vrf name mgmt protocols ospf interface veth2 network point-to-point
set vrf name mgmt protocols ospf default-information originate always

NOTE: Unless you need the default route into the mgmt VRF, you could just advertise the 10.0.2.96/27 prefix into OSPF.

You can also just use the veth1 IP as the source of the RPKI connection, and that would simplify it further.

[hanisirfan]

Never heard of veth but now I did. Thanks for pointing that out. After some tweaks, your solution works

I hope maybe in the future we can bind RPKI to certain address, interface or VRF.

[L0crian]

Just a heads up, RPKI does have a source-address option and a VRF level config in rolling. It’s just not on LTS yet.

[dhess]

We’d like to take advantage of the VRF level config for RPKI. I tried it in 2025.09.27-0018-rolling, but it didn’t seem to work. Perhaps that’s because I’d already configured the same RPKI cache the standard way and the same server can’t be configured in multiple VRFs?

Anyway, @L0crian , if RPKI is configured in a particular VRF, are the RPKI validation values for each route available in all VRFs, or only in the VRF where the RPKI cache is configured?

[L0crian]

I tried it in 2025.09.27-0018-rolling , but it didn’t seem to work. Perhaps that’s because I’d already configured the same RPKI cache the standard way and the same server can’t be configured in multiple VRFs?

You can use the same upstream server just fine. You may have something else wrong in your setup.

vyos@vyos:~$ show rpki cache-connection 
Connected to group 5
rpki tcp cache 10.0.95.83 3323 pref 5 (connected)

vyos@vyos:~$ show rpki cache-connection vrf test
Connected to group 5
rpki tcp cache 10.0.95.83 3323 pref 5 (connected)

Anyway, @L0crian , if RPKI is configured in a particular VRF, are the RPKI validation values for each route available in all VRFs, or only in the VRF where the RPKI cache is configured?

Per the FRR documentation, the cache peerings are contextual, so if you peer in a given VRF, then it’ll only consult the cache within that VRF.