PPPOE WAN Configuration issue

lepri13 · November 7, 2021, 10:47am

Hello,

I am in the process of migrating my main firewall to vyos. And I have an issue with WAN interface. I am getting an IP address however when I try to ping or trace-route from vyos I get:
Operation not permitted error.

vyos@vyos:~$ traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
send: Operation not permitted

Looks like I am getting external IP 90.XX/32, I have tried routing options for pppoe interface ‘auto’ and ‘force’ both are producing the same results.

Any input would be appreciated.

lepri13 · November 7, 2021, 10:48am

IP route:

run show ip route

Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup

S>* 0.0.0.0/0 [1/0] is directly connected, pppoe0, weight 1, 00:03:02
C>* 192.168.1.0/24 is directly connected, eth1, 00:23:15
C>* 212.XX.XX.XX/32 is directly connected, pppoe0, 00:03:03

Show Interfaces:

run show interfaces

Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description

eth0 - u/u WAN
eth1 192.168.1.10/24 u/u LAN
lo 127.0.0.1/8 u/u
::1/128
pppoe0 90.XX.XX.XX/32 u/u

Network Layout:

WAN - PPPOE modem in bridge mode
/
WAN - vyos physical eth0
/
LAN - vyos physical eth1

lepri13 · November 7, 2021, 10:52am

Configuration:

$ show configuration 
all-ping enable
    broadcast-ping disable
    config-trap disable
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians enable
    name LAN-LOCAL {
        default-action accept
    }
    name LAN-WAN {
        default-action accept
    }
    name LOCAL-LAN {
        default-action accept
    }
    name LOCAL-WAN {
        default-action accept
    }
    name WAN-LAN {
        default-action drop
        rule 5 {
            action accept
            description "Allow EST/Related Traffic"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action accept
            protocol icmp
            state {
                new enable
            }
        }
    }
    name WAN-LOCAL {
        default-action drop
        rule 5 {
            action accept
            description "Allow EST/Related Traffic"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action accept
            protocol icmp
            state {
                new enable
            }
        }
    }
    receive-redirects disable
    send-redirects enable
    source-validation disable
    syn-cookies enable
    twa-hazards-protection disable
}
interfaces {
    ethernet eth0 {
        description WAN
        hw-id XX
    }
    ethernet eth1 {
        address 192
        description LAN
        hw-id XX
    }
    loopback lo {
    }
    pppoe pppoe0 {
        authentication {
            password XX
            user XX
        }
        default-route force
        mtu 1492
        source-interface eth0
    }
}
nat {
    source {
        rule 100 {
            outbound-interface eth0
            source {
                address 192
            }
            translation {
                address masquerade
            }
        }
    }
}
service {
    dns {
        forwarding {
            allow-from 192
            cache-size 0
            listen-address 192
            name-server 192
        }
    }
    ssh {
        port 22
    }
}
system {
    config-management {
        commit-revisions 100
    }
    conntrack {
        modules {
            ftp
            h323
            nfs
            pptp
            sip
            sqlnet
            tftp
        }
    }
    console {
        device ttyS0 {
            speed 115200
        }
    }
    host-name vyos
    login {
        banner {
            post-login "Welcome"
        }
        user vyos {
            authentication {
                encrypted-password ****************
                plaintext-password ****************
            }
        }
    }
    name-server 192
    ntp {
        server time {
        }
        server time {
        }
        server time {
        }
    }
    syslog {
        global {
            facility all {
                level info
            }
            facility protocols {
                level debug
            }
        }
    }
}
zone-policy {
    zone LAN {
        default-action drop
        from LOCAL {
            firewall {
                name LOCAL-LAN
            }
        }
        from WAN {
            firewall {
                name WAN-LAN
            }
        }
        interface eth1
    }
    zone LOCAL {
        default-action drop
        from LAN {
            firewall {
                name LAN-LOCAL
            }
        }
        from WAN {
            firewall {
                name WAN-LOCAL
            }
        }
        local-zone
    }
    zone WAN {
        default-action drop
        from LAN {
            firewall {
                name LAN-WAN
            }
        }
        from LOCAL {
            firewall {
                name LOCAL-WAN
            }
        }
        interface eth0
    }
}

lepri13 · November 7, 2021, 10:55am

Apologies for spam had to break up the post as I was getting error that I can’t post more than 2 links in the post.

Base configuration was taken from:
https://blog.kroy.io/2020/05/04/vyos-from-scratch-edition-1

and pppoe configuration was taken from:

https://docs.vyos.io/en/equuleus/configuration/interfaces/pppoe.html?highlight=pppoe#operating-modes

16again · November 7, 2021, 1:21pm

Your masquerade rule should use pppoe as outbound interface , not eth0
But that won’t solve ping/traceroute issue from vyos itself.

Try adding manual default route, next-hop-interface is pppoe0
Also, try without ZBF stuff. (although Local->WAN looks OK)
Can you ping next hop 212.x.x.x ?

lepri13 · November 8, 2021, 6:54pm

I can ping 90.X.X.X but not the 212.X.X.X. also moving NAT to pppoe interface didn’t have any effect ether.
Still getting the same error.
I wonder does anyone have any sort of configuration basic pppoe configuration that works? Just want to test known good configuration.

lepri13 · November 8, 2021, 6:56pm

Forgot to mention I am on the 1.3 epa3.

lepri13 · November 8, 2021, 8:06pm

From looking at the log the only thing that jumps out at me:

Nov 08 19:21:03 zebra[988]: warning: PtP interface pppoe0 with addr 90.X/32 needs a peer address

Log:
Nov 08 19:20:53 kernel: r8169 0000:01:00.0 eth0: Link is Up - 100Mbps/Full - flow control rx/tx
Nov 08 19:20:53 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Nov 08 19:20:53 netplugd[908]: eth0: state INACTIVE flags 0x00001003 UP,BROADCAST,MULTICAST → 0x00011043 UP,BROADCAST,RUNNING,MULTICAST,10000
Nov 08 19:20:53 netplugd[3437]: /etc/netplug/netplug eth0 in → pid 3437
Nov 08 19:20:53 netplugd[908]: eth0: state INNING pid 3437 exited status 0
Nov 08 19:20:54 ntpd[2405]: bind(26) AF_INET6 fe80::%2#123 flags 0x11 failed: Cannot assign requested address
Nov 08 19:20:54 ntpd[2405]: unable to create socket on eth0 (7) for fe80::%2#123
Nov 08 19:20:54 ntpd[2405]: failed to init interface for address fe80::%2
Nov 08 19:20:56 ntpd[2405]: Listen normally on 8 eth0 [fe80::%2]:123
Nov 08 19:20:56 ntpd[2405]: new interface(s) found: waking up resolver
Nov 08 19:21:03 pppd[2277]: Send PPPOE Discovery V1T1 PADI session 0x0 length 4
Nov 08 19:21:03 pppd[2277]: dst ff:ff:ff:ff:ff:ff src 8c:X
Nov 08 19:21:03 pppd[2277]: [service-name]
Nov 08 19:21:03 pppd[2277]: Recv PPPOE Discovery V1T1 PADO session 0x0 length 41
Nov 08 19:21:03 pppd[2277]: dst 8c:X src 84:X
Nov 08 19:21:03 pppd[2277]: [AC-name WAHP06-BNG-01] [service-name] [AC-cookie X]
Nov 08 19:21:03 pppd[2277]: Send PPPOE Discovery V1T1 PADR session 0x0 length 24
Nov 08 19:21:03 pppd[2277]: dst 84:X src 8c:X
Nov 08 19:21:03 pppd[2277]: [service-name] [AC-cookie X]
Nov 08 19:21:03 pppd[2277]: Recv PPPOE Discovery V1T1 PADS session 0x8f4 length 41
Nov 08 19:21:03 pppd[2277]: dst 8c:X src 84:X
Nov 08 19:21:03 pppd[2277]: [service-name] [AC-name WAHP06-BNG-01] [AC-cookie V]
Nov 08 19:21:03 pppd[2277]: PADS: Service-Name: ‘’
Nov 08 19:21:03 pppd[2277]: PPP session is 2292
Nov 08 19:21:03 kernel: pppoe0: renamed from ppp0
Nov 08 19:21:03 pppd[2277]: Connected to 84:X via interface eth0
Nov 08 19:21:03 pppd[2277]: using channel 1
Nov 08 19:21:03 pppd[2277]: Renamed interface ppp0 to pppoe0
Nov 08 19:21:03 pppd[2277]: Using interface pppoe0
Nov 08 19:21:03 pppd[2277]: Connect: pppoe0 <–> eth0
Nov 08 19:21:03 pppd[2277]: sent [LCP ConfReq id=0x1 <mru 1492> <magic 0x>]
Nov 08 19:21:03 isisd[1017]: circuit already disconnected
Nov 08 19:21:03 bgpd[997]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp0 in VRF 0
Nov 08 19:21:03 systemd-udevd[3477]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Nov 08 19:21:03 pppd[2277]: rcvd [LCP ConfReq id=0x83 <mru 1492> <magic 0x>]
Nov 08 19:21:03 pppd[2277]: sent [LCP ConfAck id=0x83 <mru 1492> <magic 0x>]
Nov 08 19:21:03 pppd[2277]: rcvd [LCP ConfAck id=0x1 <mru 1492> <magic 0x>]
Nov 08 19:21:03 pppd[2277]: sent [LCP EchoReq id=0x0 magic=0x]
Nov 08 19:21:03 isisd[1017]: circuit already disconnected
Nov 08 19:21:03 systemd-udevd[3477]: link_config: could not get ethtool features for ppp0
Nov 08 19:21:03 systemd-udevd[3477]: Could not set offload features of ppp0: No such device
Nov 08 19:21:03 pppd[2277]: rcvd [CHAP Challenge id=0xb4 , name = “JUNOS”]
Nov 08 19:21:03 pppd[2277]: sent [CHAP Response id=0xb4 , name = “X@X”]
Nov 08 19:21:03 pppd[2277]: rcvd [LCP EchoRep id=0x0 magic=X]
Nov 08 19:21:03 pppd[2277]: rcvd [CHAP Success id=0xb4 “”]
Nov 08 19:21:03 pppd[2277]: CHAP authentication succeeded
Nov 08 19:21:03 pppd[2277]: CHAP authentication succeeded
Nov 08 19:21:03 pppd[2277]: peer from calling number 84:X authorized
Nov 08 19:21:03 pppd[2277]: sent [IPCP ConfReq id=0x1 <addr 0.0.0.0> <ms-dns1 0.0.0.0> <ms-dns2 0.0.0.0>]
Nov 08 19:21:03 pppd[2277]: rcvd [IPCP ConfReq id=0x2d <addr 212.X>]
Nov 08 19:21:03 pppd[2277]: sent [IPCP ConfAck id=0x2d <addr 212.X>]
Nov 08 19:21:03 pppd[2277]: rcvd [IPCP ConfNak id=0x1 <addr 90.X> <ms-dns1 90.X> <ms-dns2 90.X>]
Nov 08 19:21:03 pppd[2277]: sent [IPCP ConfReq id=0x2 <addr 90.X> <ms-dns1 90.X> <ms-dns2 90.255.255.90>]
Nov 08 19:21:03 pppd[2277]: rcvd [IPCP ConfAck id=0x2 <addr 90.X> <ms-dns1 90.X> <ms-dns2 90.X>]
Nov 08 19:21:03 zebra[988]: warning: PtP interface pppoe0 with addr 90.X/32 needs a peer address
Nov 08 19:21:03 pppd[2277]: Script /etc/ppp/ip-pre-up started (pid 3486)
Nov 08 19:21:03 pppd[3491]: executing /etc/ppp/ip-pre-up.d/1000-vyos-pppoe-pppoe0
Nov 08 19:21:03 pppd[2277]: Script /etc/ppp/ip-pre-up finished (pid 3486), status = 0x0
Nov 08 19:21:03 pppd[2277]: local IP address 90.X
Nov 08 19:21:03 pppd[2277]: remote IP address 212.X
Nov 08 19:21:03 pppd[2277]: primary DNS address 90.X
Nov 08 19:21:03 pppd[2277]: secondary DNS address 90.X
Nov 08 19:21:03 pppd[2277]: Script /etc/ppp/ip-up started (pid 3492)
Nov 08 19:21:03 pppd[3503]: executing /etc/ppp/ip-up.d/1000-vyos-pppoe-pppoe0
Nov 08 19:21:03 pppd[3510]: added default route via pppoe0
Nov 08 19:21:04 systemd[2547]: opt-vyatta-config-tmp-new_config_3494.mount: Succeeded.
Nov 08 19:21:04 systemd[1]: opt-vyatta-config-tmp-new_config_3494.mount: Succeeded.
Nov 08 19:21:04 pppd[2277]: Script /etc/ppp/ip-up finished (pid 3492), status = 0x0
Nov 08 19:21:05 ntpd[2405]: Listen normally on 9 pppoe0 90.X:123
Nov 08 19:21:05 ntpd[2405]: new interface(s) found: waking up resolver

RyVolodya · November 11, 2021, 2:05pm

Hello @lepri13. I reproduced your configuration in network laboratory. You need change firewall configuration:

delete zone-policy zone WAN interface 'eth0'
set zone-policy zone WAN interface 'pppoe0'

lepri13 · November 11, 2021, 2:20pm

Hey @RyVolodya.

Thank you so much. I will give it a try over the weekend and report back.

lepri13 · November 18, 2021, 7:35pm

@RyVolodya

I am still testing however I have one observation. I need to have PPPOE interface up in order to be able to configure a zone-policy.

Is there a way to enable / declared the interface active while configuring?

On the side note I was able to ping 1.1.1.1 using updated policy however my machine never got any ping replys. I will continue to troubleshoot at the weekend.

lepri13 · November 18, 2021, 7:53pm

Screenshot 2021-11-18 at 19.52.55

lepri13 · November 18, 2021, 8:12pm

Screenshot 2021-11-18 at 20.11.16

RyVolodya · November 20, 2021, 7:39am

Hello @lepri13
Please send your current config:
vyos@vyos:~$ show configuration commands | strip-private

lepri13 · November 20, 2021, 9:10am

set firewall name LAN-LOCAL default-action ‘accept’
set firewall name LAN-WAN default-action ‘accept’
set firewall name LOCAL-LAN default-action ‘accept’
set firewall name LOCAL-WAN default-action ‘accept’
set firewall name WAN-LAN default-action ‘drop’
set firewall name WAN-LAN rule 5 action ‘accept’
set firewall name WAN-LAN rule 5 description ‘Allow EST/Related Traffic’
set firewall name WAN-LAN rule 5 state established ‘enable’
set firewall name WAN-LAN rule 5 state related ‘enable’
set firewall name WAN-LAN rule 20 action ‘accept’
set firewall name WAN-LAN rule 20 protocol ‘icmp’
set firewall name WAN-LAN rule 20 state new ‘enable’
set firewall name WAN-LOCAL default-action ‘drop’
set firewall name WAN-LOCAL rule 5 action ‘accept’
set firewall name WAN-LOCAL rule 5 description ‘Allow EST/Related Traffic’
set firewall name WAN-LOCAL rule 5 state established ‘enable’
set firewall name WAN-LOCAL rule 5 state related ‘enable’
set firewall name WAN-LOCAL rule 20 action ‘accept’
set firewall name WAN-LOCAL rule 20 protocol ‘icmp’
set firewall name WAN-LOCAL rule 20 state new ‘enable’
set interfaces ethernet eth0 address ‘xxx.xxx.1.11/24’
set interfaces ethernet eth0 description ‘LAN’
set interfaces ethernet eth0 hw-id ‘XX:XX:XX:XX:XX:6c’
set interfaces ethernet eth1 description ‘WAN’
set interfaces ethernet eth1 hw-id ‘XX:XX:XX:XX:XX:c5’
set interfaces loopback lo
set interfaces pppoe pppoe0 authentication password xxxxxx
set interfaces pppoe pppoe0 authentication user xxxxxx
set interfaces pppoe pppoe0 default-route ‘auto’
set interfaces pppoe pppoe0 mtu ‘1492’
set interfaces pppoe pppoe0 source-interface ‘eth1’
set nat source rule 100 outbound-interface ‘eth1’
set nat source rule 100 source address ‘xxx.xxx.1.0/24’
set nat source rule 100 translation address ‘masquerade’
set service dns forwarding allow-from ‘xxx.xxx.1.0/24’
set service dns forwarding cache-size ‘0’
set service dns forwarding listen-address ‘xxx.xxx.1.11’
set service dns forwarding name-server ‘xxx.xxx.1.1’
set service ssh port ‘22’
set system config-management commit-revisions ‘100’
set system conntrack modules ftp
set system conntrack modules h323
set system conntrack modules nfs
set system conntrack modules pptp
set system conntrack modules sip
set system conntrack modules sqlnet
set system conntrack modules tftp
set system console device ttyS0 speed ‘115200’
set system host-name xxxxxx
set system login user xxxxxx authentication encrypted-password xxxxxx
set system login user xxxxxx authentication plaintext-password xxxxxx
set system name-server ‘xxx.xxx.1.1’
set system ntp server xxxxx.tld
set system ntp server xxxxx.tld
set system ntp server xxxxx.tld
set system syslog global facility all level ‘info’
set system syslog global facility protocols level ‘debug’
set zone-policy zone LAN default-action ‘drop’
set zone-policy zone LAN from LOCAL firewall name ‘LOCAL-LAN’
set zone-policy zone LAN from WAN firewall name ‘WAN-LAN’
set zone-policy zone LAN interface ‘eth0’
set zone-policy zone LOCAL default-action ‘drop’
set zone-policy zone LOCAL from LAN firewall name ‘LAN-LOCAL’
set zone-policy zone LOCAL from WAN firewall name ‘WAN-LOCAL’
set zone-policy zone LOCAL local-zone
set zone-policy zone WAN default-action ‘drop’
set zone-policy zone WAN from LAN firewall name ‘LAN-WAN’
set zone-policy zone WAN from LOCAL firewall name ‘LOCAL-WAN’
set zone-policy zone WAN interface ‘pppoe0’