Config lost on upgrade + can't use negating

IZT_crobinson · June 8, 2020, 2:16pm

Hello on 1.3-rolling-202006070117 upgrade it seems I lost my NAT rules. So I went to the /config/config.boot.2020-06-07-194824.pre-migration and got my nat rules back. However, I can’t commit because apparently ! isn’t accepted anymore?

VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Make sure you are running the latest version of the code available at
  https://downloads.vyos.io/rolling/current/amd64/vyos-rolling-latest.iso
- Consult the forum to see how to handle this issue
  https://forum.vyos.io
- Join our community on slack where our users exchange help and advice
  https://vyos.slack.com

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your
  business policy requires it)
- and include all the information presented below

Report Time:      2020-06-08 10:09:11
Image Version:    VyOS 1.3-rolling-202006070117
Release Train:    equuleus

Built by:         autobuild@vyos.net
Built on:         Sun 07 Jun 2020 01:17 UTC
Build UUID:       d258ac65-52c3-4b94-87c7-60e4a6de29c1
Build Commit ID:  972534c08225bc

Architecture:     x86_64
Boot via:         installed image
System type:      VMware guest

Hardware vendor:  VMware, Inc.
Hardware model:   VMware Virtual Platform
Hardware S/N:     VMware-XX
Hardware UUID:    XX

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/nat.py", line 268, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/nat.py", line 256, in apply
    cmd(f'{iptables_nat_config}')
  File "/usr/lib/python3/dist-packages/vyos/util.py", line 178, in cmd
    raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: /tmp/vyos-nat-rules.nft
returned:
exit code: 1

noteworthy:
cmd '/tmp/vyos-nat-rules.nft'
returned (out):

returned (err):
/tmp/vyos-nat-rules.nft:34:94-94: Error: syntax error, unexpected !
add rule ip nat PREROUTING iifname "eth0" ip protocol tcp ip daddr XX.XX.XX.XX tcp dport { !22 } counter dnat to 172.16.50.15 comment "DST-NAT-1 tcp     _udp"
                                                                                             ^
/tmp/vyos-nat-rules.nft:37:94-94: Error: syntax error, unexpected !
add rule ip nat PREROUTING iifname "eth0" ip protocol udp ip daddr XX.XX.XX.XX udp dport { !22 } counter dnat to 172.16.50.15 comment "DST-NAT-1 tcp     _udp"
                                                                                             ^

[[nat]] failed
Commit failed
[edit]

c-po · June 8, 2020, 4:01pm

Please provide your NAT configuration for reproducibility.

IZT_crobinson · June 8, 2020, 4:55pm

In all the configs, it seems that port !22 is what’s causing the issue

c-po · June 8, 2020, 5:48pm

Please paste the appropriate commands from show configuration commands here so it can easily be reproduced.

Bug opened: ⚓ T2571 NAT destination port with ! results in error

IZT_crobinson · June 9, 2020, 4:30pm

@c-po I send you a pm with the information.

blackhole · June 11, 2020, 12:02am

This is a bugger. I just updated to “1.3-rolling-202006101523” and was just about to post a topic on this but seems you have one

My issue is exactly same but for IP range, I am using !192.168.67.243-192.168.67.244
It seems nat rules are not accepting “!” and like you I have also lost all my NAT rules as a result, same errors as you above when manually trying to commit as well.

I have added my information to the phabricator as well.
Is it worth mentioning it is failing to migrate on reboot as well as a manual commit.

androme · June 11, 2020, 1:41pm

Hello, same problem for me with VyOS 1.3-rolling-202006090117 and VyOS 1.3-rolling-202006110117

c-po · June 12, 2020, 12:13pm

Fixed in recent rolling release

blackhole · June 12, 2020, 12:42pm

Awesome, thanks!

Hopefully try it out either tonight or in morning and report back

blackhole · June 13, 2020, 3:10pm

Hi c-po, the fix for this works great, NAT rules are not being problematic at all.
However, I am getting an issue now with latest rolling that DHCP relay (dont have DHCP running on VyOS) does not appear to be working all of a sudden. Not sure if related but suspect it is not.

Nothing in the logs apart from this:

Jun 14 01:04:20 vyos007 dhcrelay[17517]: Discarding packet received on eth1.11v11 interface that has no IPv4 address assigned.
Jun 14 01:04:21 vyos007 dhcrelay[17517]: Discarding packet received on eth2 interface that has no IPv4 address assigned.
Jun 14 01:04:21 vyos007 dhcrelay[17517]: Discarding packet received on eth1 interface that has no IPv4 address assigned.
Jun 14 01:04:21 vyos007 dhcrelay[17517]: Discarding packet received on eth0 interface that has no IPv4 address assigned.
Jun 14 01:04:31 vyos007 dhcrelay[17517]: Discarding packet received on eth2.7v7 interface that has no IPv4 address assigned.

Which is strange considering…

mario@vyos007:~$ show interfaces vrrp
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0.17v17       192.168.17.253/24                 u/u
eth0.67v67       192.168.67.253/24                 u/u
eth0.79v79       192.168.79.253/24                 u/u
eth1.11v11       192.168.11.253/24                 u/u
eth1.13v13       192.168.13.253/24                 u/u
eth1.131v131     192.168.131.253/24                u/u
eth2.7v7         192.168.7.253/24                  u/u
eth2.53v53       192.168.53.253/24                 u/u

I even restarted dhcp-relay service but to no avail, do not see the usual request/offer for DHCP interestingly enough on 2 of the VLAN’s I have DHCP relay running on

My actual DHCP relay config is and it has not changed over a long time, confirmed nothing too odd via show log after migration or subsequent log that stood out in relation to the DHCP settings I have below.

set service dhcp-relay interface 'eth1.131v131'
set service dhcp-relay interface 'eth0.67v67'
set service dhcp-relay interface 'eth1.13v13'
set service dhcp-relay interface 'eth1.11v11'
set service dhcp-relay relay-options relay-agents-packets 'discard'
set service dhcp-relay server '192.168.67.241'
set service dhcp-relay server '192.168.67.242'

blackhole · June 13, 2020, 3:19pm

I wonder if it is related to this perhaps ⚓ T2576 "show interfaces" does not return VTI

So, since I am reverting back to my last known working image 1.3-rolling-202005130117 I decided to test show interfaces in 1.3-rolling-202006120643

mario@vyos007:~$ show system image
The system currently has the following image(s) installed:

   1: 1.3-rolling-202006120643 (default boot) (running image)
   2: 1.3-rolling-202005130117

mario@vyos007:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             -                                 u/u
eth0.167         [REMOVED]                    u/u  WAN
eth0.17          192.168.17.252/24                 u/u  Public
eth0.67          192.168.67.252/24                 u/u  DMZ
eth0.79          192.168.79.252/24                 u/u  Download
eth1             -                                 u/u
eth1.11          192.168.11.252/24                 u/u  IOT
eth1.13          192.168.13.252/24                 u/u  LAN
eth1.131         192.168.131.252/24                u/u  Guest
eth2             -                                 u/u
eth2.53          192.168.53.252/24                 u/u  Cam
eth2.7           192.168.7.252/24                  u/u  Management
lo               127.0.0.1/8                       u/u
                 ::1/128

Well, I am back on the old image and DHCP is working fine through the relay immediately, did the same commands as above and sure enough, maybe T2576 is related?

mario@vyos007:~$ show system image
The system currently has the following image(s) installed:

   1: 1.3-rolling-202006120643
   2: 1.3-rolling-202005130117 (default boot) (running image)

mario@vyos007:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             -                                 u/u
eth0.17          192.168.17.252/24                 u/u  Public
eth0.17v17       192.168.17.253/24                 u/u
eth0.67          192.168.67.252/24                 u/u  DMZ
eth0.67v67       192.168.67.253/24                 u/u
eth0.79          192.168.79.252/24                 u/u  Download
eth0.79v79       192.168.79.253/24                 u/u
eth0.167         [REMOVED]                    u/u  WAN
eth1             -                                 u/u
eth1.11          192.168.11.252/24                 u/u  IOT
eth1.11v11       192.168.11.253/24                 u/u
eth1.13          192.168.13.252/24                 u/u  LAN
eth1.13v13       192.168.13.253/24                 u/u
eth1.131         192.168.131.252/24                u/u  Guest
eth1.131v131     192.168.131.253/24                u/u
eth2             -                                 u/u
eth2.7           192.168.7.252/24                  u/u  Management
eth2.7v7         192.168.7.253/24                  u/u
eth2.53          192.168.53.252/24                 u/u  Cam
eth2.53v53       192.168.53.253/24                 u/u
lo               127.0.0.1/8                       u/u
                 ::1/128

Have raised issue under ⚓ T2592 dhcp-relay discarding packets on valid interfaces