VyOS and Cable Modems - Can't get DHCP lease from modem - Works for Direct Connect PC

General questions
1

Hello,

This one is a bit odd. I’ve been running VyOS 1.5 nightlies for a while now but recently ran into a problem with getting my VyOS router to take a DHCP lease from a new cable modem.

My previous modems have both been Motorola/Arris Surfboards, but this new one is A Netgear Nighthawk CM2000. For whatever reason, connecting a PC directly to the new modem is able to get a lease and get onto the internet, but the moment I have it fully reset and try to connect my VyOS router to it, it won’t pick up a DHCP lease for the life of me.

TCP Dump on the WAN interface does see outbound DHCP requests, but never sees anything back. I’m not sure if it’s just something weird with VyOS’s DHCP implementation or if it’s the modem that is the problem.

Again, previous modems have worked flawlessly with my configuration.The only difference here is a new modem from a different vendor (Netgear). I’ll go to Netgear support if suggested, but figured I would ask if anyone has had a similar experience.

Anyone experience similar issues? Any suggestions with how to proceed? This one is frustrating.

Let me know your thoughts. Thanks!

2

I‘ve heard from ISPs that only support a single MAC on the cable link.

You might need to either wait until it times out, or use the mac CLI command on the ethernet interface to set it to the same MAC as your PC

3

Yeah, I thought of that already and even cloned the mac of the device that did get a DHCP lease without issue, it still wouldn’t get a lease.

I think I’ll have to reach out to Netgear support for this one. It is definitely a strange issue.
Thanks!

5

What happens if you kick the dhcp client on VyOS after the modem has been up for five minutes or so?

I had a problem a few years ago when Spectrum forced me to replace my modem and the LAN interface would come up immediately, but the modem hasn’t completed its authorization and negotiation with the CMT. The dhcp client would time out, but would eventually get an IP after a back-off period.

6

A little more testing has been performed. I went to Xfinity and got their modem and activated it and put it into bridge mode. I am not able to get a DHCP lease on any recent 1.5 nightlies. Currently I have 1.5-rolling-202407070023 (previously 1.5-rolling-202405220019) as active.

Funny enough, when I roll back to a previous image (1.5-rolling-202311210100) I am able to get a DHCP lease without issue. It’s incredibly weird. As far as I can tell, TCPdump looks the same for the DHCP datagrams.

What changes, if any, have taken place to the DHCP client between 202311210100 and now that could be impacting this? If I have to, I will roll back to the older version but that would not be preferred since there’s some critical bugs/vulns that are likely fixed in the newer releases.

7

So just to be clear, in bridge mode, the ISP is still enabling DHCP. If you connect the same PC (dangerous), does it get a DHCP lease from the ISP?

8

That is correct. I can get a lease with anything else with no issue. But when running the two previously mentioned newer rolling images, it never gets a response. I can see the request going out, but no response comes back. I’m at a bit of a loss as to what the issue is, especially since my laptop has (with firewall on!) no problem getting a DHCP lease, and when I’m running the older build, the VyOS router has no problem either.

I went ahead and compared the previous rolling version and the newer version and the only changes i’m really seeing are some changes to DHCP server configuration syntax. Everything else seems to be the same. Here’s a heavily sanitized copy of the configuration of the router from 1.5-rolling-202407070023 which is the one that is unable to get a DHCP lease for the WAN interface.

Any thoughts would be greatly appreciated. Thanks.

firewall {
    global-options {
        all-ping "enable"
        broadcast-ping "disable"
        ip-src-route "disable"
        ipv6-receive-redirects "disable"
        ipv6-src-route "disable"
        log-martians "enable"
        receive-redirects "disable"
        send-redirects "enable"
        source-validation "disable"
        syn-cookies "enable"
    }
    ipv4 {
        forward {
            filter {
                default-action "accept"
                rule 5 {
                    action "jump"
                    inbound-interface {
                        name "eth0"
                    }
                    jump-target "WAN_IN"
                }
                rule 7 {
                    action "accept"
                    connection-status {
                        nat "destination"
                    }
                    state "new"
                }
                rule 10 {
                    action "accept"
                    inbound-interface {
                        name "vti0"
                    }
                }
                rule 11 {
                    action "accept"
                    inbound-interface {
                        name "vti1"
                    }
                }
            }
        }
        input {
            filter {
                default-action "accept"
                rule 5 {
                    action "jump"
                    inbound-interface {
                        name "eth0"
                    }
                    jump-target "WAN_LOCAL"
                }
            }
        }
        name WAN_IN {
            default-action "drop"
            default-log
            description "WAN to internal"
            rule 10 {
                action "accept"
                description "Allow established/related"
                state "established"
                state "related"
            }
            rule 20 {
                action "drop"
                description "Drop invalid state"
                state "invalid"
            }
            rule 30 {
                action "accept"
                description "app-inbound"
                destination {
                    address "x.x.x.10"
                    port "42420"
                }
                protocol "tcp"
            }
            rule 31 {
                action "accept"
                destination {
                    address "x.x.x.10"
                    port "32400"
                }
                protocol "tcp"
            }
            rule 40 {
                action "accept"
                description "Game"
                destination {
                    address "x.x.x.88"
                    port "5200"
                }
                protocol "tcp"
            }
            rule 50 {
                action "accept"
                destination {
                    address "x.x.x.88"
                    port "42413"
                }
                protocol "tcp"
            }
        }
        name WAN_LOCAL {
            default-action "drop"
            default-log
            description "WAN to router"
            rule 10 {
                action "accept"
                description "Allow established/related"
                state "established"
                state "related"
            }
            rule 20 {
                action "drop"
                description "Drop invalid state"
                state "invalid"
            }
            rule 30 {
                action "accept"
                description "SSH"
                destination {
                    port "22"
                }
                protocol "tcp"
            }
            rule 31 {
                action "accept"
                destination {
                    port "42420"
                }
                protocol "tcp"
            }
        }
    }
    ipv6 {
        forward {
            filter {
                default-action "accept"
                rule 5 {
                    action "jump"
                    inbound-interface {
                        name "eth0"
                    }
                    jump-target "WAN6_IN"
                }
            }
        }
        name WAN6_IN {
            default-action "drop"
            description "WAN6 to internal"
            rule 10 {
                action "accept"
                description "Allow established/related"
                state "established"
                state "related"
            }
            rule 20 {
                action "drop"
                description "Drop invalid state"
                state "invalid"
            }
        }
        name WAN6_LOCAL {
            default-action "drop"
            description "WAN6 to router"
            rule 10 {
                action "accept"
                description "Allow established/related"
                state "established"
                state "related"
            }
            rule 20 {
                action "drop"
                description "Drop invalid state"
                state "invalid"
            }
        }
    }
}
interfaces {
    bridge br0 {
        address "x.x.x.1/24"
        description "Local"
        ipv6 {
            address {
                autoconf
            }
        }
        member {
            interface eth1 {
            }
            interface eth2 {
            }
            interface eth3 {
            }
            interface eth4 {
            }
        }
        mtu "1500"
    }
    ethernet eth0 {
        address "dhcp"
        address "dhcpv6"
        dhcp-options {
            mtu
        }
        dhcpv6-options {
            pd 0 {
                interface br0 {
                    address "1"
                }
                length "64"
            }
        }
        duplex "auto"
        hw-id "xx:xx:xx:1e:39:50"
        ip
        ipv6 {
            address {
                autoconf
            }
        }
        speed "auto"
    }
    ethernet eth1 {
        duplex "auto"
        hw-id "xx:xx:xx:07:4a:70"
        speed "auto"
    }
    ethernet eth2 {
        duplex "auto"
        hw-id "xx:xx:xx:07:4a:71"
        speed "auto"
    }
    ethernet eth3 {
        duplex "auto"
        hw-id "xx:xx:xx:07:4a:72"
        speed "auto"
    }
    ethernet eth4 {
        duplex "auto"
        hw-id "xx:xx:xx:07:4a:73"
        speed "auto"
    }
    loopback lo {
    }
    vti vti0 {
        address "x.x.x.1/31"
        address "xxxx:xxxx:xxxx:ffff::fffd/127"
        description "to-vpn1"
        ipv6 {
            address
        }
        mtu "1408"
    }
    vti vti1 {
        address "x.x.x.3/31"
        address "xxxx:xxxx:xxxx:ffff::ffff/127"
        description "to-vpn2"
        ipv6 {
            address
        }
        mtu "1408"
    }
}
nat {
    destination {
        rule 1 {
            description "App"
            destination {
                port "42420"
            }
            inbound-interface {
                name "eth0"
            }
            protocol "tcp"
            translation {
                address "x.x.x.10"
                port "32400"
            }
        }
        rule 2 {
            description "Game"
            destination {
                port "5200"
            }
            inbound-interface {
                name "eth0"
            }
            protocol "tcp"
            translation {
                address "x.x.x.88"
                port "5200"
            }
        }
        rule 3 {
            description "App New"
            destination {
                port "42424"
            }
            inbound-interface {
                name "eth0"
            }
            protocol "tcp"
            translation {
                address "x.x.x.10"
                port "32400"
            }
        }
        rule 4 {
            description "App New 2"
            destination {
                port "42413"
            }
            inbound-interface {
                name "eth0"
            }
            protocol "tcp"
            translation {
                address "x.x.x.88"
                port "42413"
            }
        }
    }
    source {
        rule 65535 {
            outbound-interface {
                name "eth0"
            }
            source {
                address "x.x.x.0/20"
            }
            translation {
                address "masquerade"
            }
        }
    }
}
<SNIP>
protocols {
    static {
        route x.x.x.0/24 {
            description "lab mgmt net"
            next-hop x.x.x.5 {
                disable
                distance "1"
            }
        }
        route x.x.x.0/24 {
            description "lab lan net"
            next-hop x.x.x.5 {
                disable
                distance "1"
            }
        }
    }
}
service {
    dhcp-server {
        listen-interface "br0"
        shared-network-name LAN {
            authoritative
            subnet x.x.x.0/24 {
                lease "86400"
                option {
                    bootfile-name "pxe-boot/lpxelinux.0"
                    bootfile-server "x.x.x.3"
                    default-router "x.x.x.1"
                    domain-name "domain.org"
                    name-server "x.x.x.225"
                    tftp-server-name "x.x.x.3"
                }
                range LAN-SCOPE {
                    start "x.x.x.20"
                    stop "x.x.x.223"
                }
                static-mapping 2FL-WIFI {
                    ip-address "x.x.x.254"
                    mac "xx:xx:xx:c3:12:38"
                }
                static-mapping 10VM {
                    ip-address "x.x.x.146"
                    mac "xx:xx:xx:04:40:72"
                    option {
                        bootfile-name "uefi64/ipxe.efi"
                    }
                }
                static-mapping Printer {
                    ip-address "x.x.x.6"
                    mac "xx:xx:xx:35:e6:92"
                }
                static-mapping PC {
                    ip-address "x.x.x.88"
                    mac "xx:xx:xx:0c:7a:56"
                }
                static-mapping PC2 {
                    ip-address "x.x.x.166"
                    mac "xx:xx:xx:6e:ce:36"
                }
                static-mapping Polycom {
                    ip-address "x.x.x.43"
                    mac "xx:xx:xx:44:8b:4e"
                }
                static-mapping WorkPC {
                    ip-address "x.x.x.150"
                    mac "xx:xx:xx:e8:ea:73"
                }
                static-mapping LAB-RT {
                    ip-address "x.x.x.5"
                    mac "xx:xx:xx:21:29:68"
                }
                subnet-id "1"
            }
        }
    }
    lldp {
        interface br0 {
        }
        interface eth0 {
            disable
        }
    }
    ntp {
        allow-client {
            address "0.0.0.0/0"
            address "::/0"
        }
        server 0.pool.ntp.org {
        }
        server 1.pool.ntp.org {
        }
        server 2.pool.ntp.org {
        }
        server 3.pool.ntp.org {
        }
    }
    router-advert {
        interface br0 {
            hop-limit "64"
            interval {
                max "600"
                min "3"
            }
            prefix ::/64 {
                valid-lifetime "2592000"
            }
            reachable-time "30000"
            retrans-timer "0"
        }
    }
    ssh {
        dynamic-protection {
            allow-from "x.x.x.0/24"
        }
        listen-address "x.x.x.1"
        port "22"
    }
}
system {
    acceleration
    config-management {
        commit-revisions "100"
    }
    conntrack {
        expect-table-size "65536"
        hash-size "65536"
        modules {
            ftp
            h323
            nfs
            pptp
            sip
            sqlnet
            tftp
        }
        table-size "524288"
    }
    console {
        device ttyS0 {
            speed "115200"
        }
    }
    domain-name "domain.org"
    host-name "gatekeeper"
    login {
        user admin {
            authentication {
                encrypted-password "<REDACTED>"
                plaintext-password ""
            }
        }
    }
    name-server "8.8.8.8"
    name-server "8.8.4.4"
    syslog {
        global {
            facility all {
                level "info"
            }
            facility local7 {
                level "debug"
            }
        }
    }
    time-zone "CST6CDT"
}
vpn {
<SNIP>
}


// Warning: Do not remove the following line.
// vyos-config-version: "bgp@5:broadcast-relay@1:cluster@2:config-management@1:conntrack@5:conntrack-sync@2:container@2:dhcp-relay@2:dhcp-server@11:dhcpv6-server@5:dns-dynamic@4:dns-forwarding@4:firewall@16:flow-accounting@1:https@6:ids@1:interfaces@33:ipoe-server@3:ipsec@13:isis@3:l2tp@9:lldp@2:mdns@1:monitoring@1:nat@8:nat66@3:ntp@3:openconnect@3:openvpn@2:ospf@2:pim@1:policy@8:pppoe-server@10:pptp@5:qos@2:quagga@11:reverse-proxy@1:rip@1:rpki@2:salt@1:snmp@3:ssh@2:sstp@6:system@27:vrf@3:vrrp@4:vyos-accel-ppp@2:wanloadbalance@3:webproxy@2"
// Release version: 1.5-rolling-202407070023
9

Can you please capture some DHCP traffic using tcpdump from both a working image and then a non-working image? Please use -w to save to a file, not ASCII pastes.

Then we could look at the two traffic captures and see what’s different about them as a potential pinpoint of the issue.

1 Like
10

sounds good to me. I’ve captured them, but how should I share them? Forum doesn’t allow their upload.

11

If there is a preferred file sharing service like dropbox,gdrive, or something like that I can share that way. I used to have access to Slack years ago but it seems the admins have done away with Slack for community members.

12

Note: Anonymized with TraceWrangler. If this seems to cause an issue let me know and i’ll see if we have other options.

1.5-rolling-202407070023 - Captured after reboot and cable modem/gateway power-cycle. No lease obtained.

1.5-rolling-202311210100 - Captured after reboot and cable modem powercycle. Lease obtained.

13

I had a good look at both those and yea, I have to agree, I can’t see why one would work and one wouldn’t. I recall seeing discussion about offloads being on/off in later version of Vyos, prehaps check what Ethernet Offload settings are active on the different Vyos versions, maybe that’s playing into it?

But yes, that aside I’m out of ideas sorry!

14

Have you tried stripping back to a very minimal config, perhaps only eth0 configured without any firewalling, NAT etc?

It might be worth a try. If it does get DHCP, then gradually add the remainder of the config until it breaks…

15

The one thing that strikes me as strange is in the newer example you receive a REQUEST without an OFFER. I’m not sure how that is possible.
And the REQUESTs do ask for an IP Address as you can see (and probably already know) in the attached screenshots.

DHCPDiscovers
DHCPDiscovers1920×1080 271 KB

DHCPRequests
DHCPRequests1920×1080 251 KB

16

I reviewed the original (not anonymized) pcap and that request / mac is coming from the cable gateway itself. You’ll notice the MAC in packet 7 and 8 is different from packet 6. The MAC ending in 70:94 is my vyos router, everything else is either coming from the CMTS or is the Cable gateway. It does seem that the anonymizer was consistent with it’s efforts, so the anonymized MAC and IP should be the same in both captures.

I will be testing out the other part (minimal config) later tonight. I may also brave tech support and do another factory reset on the modem / reactivate it with support depending on how much I want to punish myself. :crazy_face: More to come there.

17

DORA exchange look fine to me. However, supplied gateway seems invalid: GW is not in the same subnet as offered IP address.

18

For what it’s worth, I signed up for Spectrum when my AT&T fiber was down for 10 days. I tried using a VLAN on my Unifi switch to connect it to Vyos and nothing worked. I finally ended up using an unused interface directly on one of my XCP-NG servers… and for some reason, that worked.

I had the problem with VyOS 1.4-rolling-202401141908 (very nearly final), right away, but I upgraded to VyOS 1.5-rolling-202408230022 and it has had the same behavior.

I did finally get it working going from the Spectrum modem to a Unifi 16 port then TP Llink 8 port over a VLAN to the XCP-NG server in my office. But I was gone over vacation and Spectrum buried the cable… and when I came back I could not get it to work. I set my MacBook Pro to use the same VLAN from my office, taking the same path and it got DHCP from the "Border Router :stuck_out_tongue: I guess I didn’t try seeing if it could get a DHCP from Spectrum’s modem.

I don’t have any tcpdumps handy, but I can say this. When Vyos first reboots, it uses DHCP and gets a 192.168.x.x addresses. A minute or two later, it resets DHCP and then my Vyos gets a 24.70.x.x./20 address.

I saw this with my MacBook too… It initially gets a DHCP address of 192.168… then a minute later DHCP resets and it gets the direct IP.

My ATT fiber is working again, and their annoying “modem” lets me get as many DHCP addresses as I want. I can share a tcpdump if anyone is interested, I just want to add this tidbit.