Issues with default route on Starlink

Hi,

I have multihomed network with 2 internet connections. Up to not long ago it was cable and DSL but now I’m trying to replace DSL with Starlink.

Starlink had two modes of operation:

  • All in one with Wi-Fi router enabled that allocates IPv4 addresses via DHCP from 198.168.1.0/24 range
  • Router bypass mode that alocates IPv4 addresses from 100.64.0.0/10 range (CGNAT)

For obvious reasons I’m more interested in the latter one. The configuration shod work identically to my DSL connection but for some strange reasons it doesn’t.

Here are two problems that are likely related:

  • I cannot suppress the default route allocated by Starlink DHCP server
  • the default rote injected with distance of 1 that I cannot override either

And here are some more details:

r24:~$ show version

Version:          VyOS 1.3.0-epa3
Release train:    equuleus

Built by:         Sentrium S.L.
Built on:         Sun 31 Oct 2021 17:38 UTC
Build UUID:       383e45ad-b32a-4359-8183-9baacc8e69d9
Build commit ID:  bb511522cc3bb2-dirty

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (Q35 + ICH9, 2009)
Hardware S/N:     
Hardware UUID:    18a48ed6-0124-41b0-a4f8-bbbca875d989

Copyright:        VyOS maintainers and contributors

the interface config:

r24# show interfaces ethernet eth4
 address dhcp
 description WAN-STARLINK
 dhcp-options {
     no-default-route
 }
 hw-id 00:1b:21:8c:bd:a3
 vrf STARLINK
[edit]

the route table:

r24:~$ show ip route vrf STARLINK 
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

VRF STARLINK:
S>* 0.0.0.0/0 [1/0] via 100.64.0.1, eth4, weight 1, 10:30:21
K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 10:32:18
S>* 34.120.255.244/32 [1/0] is directly connected, eth4, weight 1, 10:30:21
C>* 100.64.0.0/10 is directly connected, eth4, 10:30:21
S>* 192.168.100.1/32 [1/0] is directly connected, eth4, weight 1, 10:30:21

and dhcp client log:

r24:~$ show log dhcp client
...
Apr 30 22:35:44 systemd[1]: Starting DHCP client on eth4...
Apr 30 22:35:44 systemd[1]: Started DHCP client on eth4.
Apr 30 22:35:44 dhclient-script-vyos[26939]: Current dhclient PID: 26938, Parent PID: 26937, IP version: 4, All dhclients for interface eth4: 26937 26938
Apr 30 22:35:44 dhclient-script-vyos[26939]: Passing command to /usr/sbin/ip: "link set dev eth4 up"
Apr 30 22:35:44 dhclient-script-vyos[26939]: No changes to apply via vyos-hostsd-client
Apr 30 22:35:44 dhclient[26938]: DHCPDISCOVER on eth4 to 255.255.255.255 port 67 interval 7
Apr 30 22:35:44 dhclient[26938]: DHCPOFFER of 100.123.57.53 from 100.64.0.1
Apr 30 22:35:44 dhclient[26938]: DHCPREQUEST for 100.123.57.53 on eth4 to 255.255.255.255 port 67
Apr 30 22:35:44 dhclient[26938]: DHCPACK of 100.123.57.53 from 100.64.0.1
Apr 30 22:35:44 dhclient-script-vyos[26961]: Current dhclient PID: 26938, Parent PID: 1, IP version: 4, All dhclients for interface eth4: 26938
Apr 30 22:35:44 dhclient-script-vyos[26961]: Passing command to /usr/sbin/ip: "-4 addr add 100.123.57.53/255.192.0.0 broadcast 100.127.255.255 valid_lft 300 preferred_lft 300 dev eth4 label eth4"
Apr 30 22:35:44 dhclient-script-vyos[26961]: Passing command to /usr/sbin/ip: "link set dev eth4 mtu 1500"
Apr 30 22:35:44 dhclient-script-vyos[26961]: Deleting nameservers with tag "dhcp-eth4" via vyos-hostsd-client
Apr 30 22:35:44 dhclient-script-vyos[26961]: Adding nameservers "1.1.1.1 8.8.8.8" with tag "dhcp-eth4" via vyos-hostsd-client
Apr 30 22:35:44 dhclient-script-vyos[26961]: Applying changes via vyos-hostsd-client
Apr 30 22:35:44 dhclient-script-vyos[26961]: No changes to apply via vyos-hostsd-client
Apr 30 22:35:44 dhclient-script-vyos[26961]: FRR status: running
Apr 30 22:35:44 dhclient-script-vyos[26961]: Checking if the route presented in kernel: 192.168.100.1/32 dev eth4
Apr 30 22:35:44 dhclient-script-vyos[26961]: Converted vtysh command: "ip route 192.168.100.1/32  eth4 tag 210  vrf STARLINK"
Apr 30 22:35:44 dhclient-script-vyos[26961]: Sending command to vtysh
Apr 30 22:35:44 dhclient-script-vyos[26961]: FRR status: running
Apr 30 22:35:44 dhclient-script-vyos[26961]: Checking if the route presented in kernel: 34.120.255.244/32 dev eth4
Apr 30 22:35:44 dhclient-script-vyos[26961]: Converted vtysh command: "ip route 34.120.255.244/32  eth4 tag 210  vrf STARLINK"
Apr 30 22:35:44 dhclient-script-vyos[26961]: Sending command to vtysh
Apr 30 22:35:44 dhclient-script-vyos[26961]: FRR status: running
Apr 30 22:35:44 dhclient-script-vyos[26961]: Checking if the route presented in kernel: 0.0.0.0/0 via 100.64.0.1 dev eth4
Apr 30 22:35:44 dhclient-script-vyos[26961]: Converted vtysh command: "ip route 0.0.0.0/0 100.64.0.1 eth4 tag 210  vrf STARLINK"
Apr 30 22:35:44 dhclient-script-vyos[26961]: Sending command to vtysh
Apr 30 22:35:45 dhclient[26938]: bound to 100.123.57.53 -- renewal in 150 seconds.

Additional observations:

  • The interface runs in a separate vrf but behaviour is identical in the default vrf as well. I’m exploring option of isolating starling routes
  • DHCP client log shows that route is injected with ... tag 210 which I assume is the distance but it ends up with distance of 1 in the routing table.
  • Also, if I switch Starlink into the default mode with Wi-Fi router enabled, it allocates an address and routes with correct distance. In theory I could use it as workaround (even though double NAT does not inspire confidence) bu the bigger problem is that it allocates address from 192.168.1.0/24 range that conflict with one of my existing subnets, so it is a no-go.

Any ideas? Is CGNAT address range 100.64.0.0/10 is treated in any special way by dhcp client or Vyos? Would it be possible for Starlink to use some DHCP protocol trickery to force dhcp client to behave in this way? I’m running out of ideas, so any help is appreciated.

I’ve also reported this issue here - ⚓ T4405 DHCP client sometimes ignores `no-default-route` option of an interface