Vyos 1.3 can't save config

[echowings]

Hi All,
Recently I met a problem that is this:

vyos@my-vyos# save
Saving configuration to '/config/config.boot'...
Can not open file '/opt/vyatta/etc/config/config.boot': Read-only file system

 vyos@my-vyos# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            2.0G     0  2.0G   0% /dev
tmpfs           395M   41M  354M  11% /run
/dev/vda1        20G  2.8G   16G  15% /usr/lib/live/mount/persistence
/dev/loop0      306M  306M     0 100% /usr/lib/live/mount/rootfs/1.3.4.squashfs
tmpfs           2.0G     0  2.0G   0% /usr/lib/live/mount/overlay
overlay         2.0G   30M  1.9G   2% /
tmpfs           2.0G   84K  2.0G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           2.0G  200K  2.0G   1% /var/tmp
tmpfs           2.0G  596K  2.0G   1% /tmp
none            2.0G 1000K  2.0G   1% /opt/vyatta/config
tmpfs           395M     0  395M   0% /run/user/1003
unionfs-fuse    2.0G 1000K  2.0G   1% /opt/vyatta/config/tmp/new_config_6737

[tjh]

Have you been doing any commands as root?
For example especially trying to run scripts that change things on VRRP etc transition?

show kernel log might give you some helpful information as to what has happened if there’s been a disk problem causing the filesystem to go read-only.

What does mount | grep config show you?

[echowings]

Nope. Never do anything.

[echowings]

with show log all | grep kernel
got error like these:

vyos@my-server:~$ show log all | grep kernel
Nov  7 10:59:04 my-server kernel: [41845.821307] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:22 my-server kernel: [42283.771261] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:23 my-server kernel: [42285.362525] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:24 my-server kernel: [42286.507513] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:25 my-server kernel: [42287.631400] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:27 my-server kernel: [42289.141666] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:28 my-server kernel: [42290.357088] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:29 my-server kernel: [42291.587482] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:31 my-server kernel: [42293.065976] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:32 my-server kernel: [42294.057771] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:33 my-server kernel: [42295.058422] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2
Nov  7 11:06:35 my-server kernel: [42297.112559] overlayfs: failed to resolve '/boot/1.3.4-new/rw': -2

[tjh]

Based on those errors I’d assume you’ve had some sort of disk error.
Is this running a VM, have you migrated the instance or something similar?

Best course of action might be to copy the configuration off the box using

tim@ferrari# save
Possible completions:
  <Enter>                               Save to system config file
  <file>                                Save to file on local machine
  scp://<user>:<passwd>@<host>:/<file>  Save to file on remote machine
  sftp://<user>:<passwd>@<host>/<file>  Save to file on remote machine
  ftp://<user>:<passwd>@<host>/<file>   Save to file on remote machine
  tftp://<host>/<file>                  Save to file on remote machine

so you’ve got the config backed up somewhere, then reboot.

And try to figure out what happened to your router to cause it to have disk problems.

[Apachez]

Do you use a custom VyOS version?

[echowings]

Not cusom version. Actually I install VyOS in kvm. then dd create the disk to recovery on vps. It works for a long time. But recently days the vps host has hardware error. After they recoveried the vps. it has such error like these. Maybe I might try to restore the vps with vyos again.

[Apachez]

You should be able to recover older config from the persistence path (each installation got its own subdir which is copied when system upgrade occurs so the old stuff remains untouched) and most likely do a fresh install if possible.

My question regarding custom version is that your overlayfs sseems to want to go for “/boot/1.3.4-new/rw” which looks odd to me (but I have only been using 1.4 and 1.5 so perhaps thats a 1.3 thingy to have the path named that way).

[echowings]

Thx @tjh
It really a disk error. The /config is empty except auth folder . no others folders or files in it.
Today I fixied by running these command :

#Step 1  Fix disk error  with fsck
sudo fsck -fy /dev/vda1

# Step 2 Reboot the VyOS
reboot now

Thanks guys!

[tjh]

Great news, good to know you got it sorted

[echowings]

The error found by that I try to upgrade vyos 1.3.4 version to fix this problem. But no can do. then I got the error like these.