I’ve been running a number of Vyatta routers that were later upgrade to VyOS. Things have been working great with them until I upgraded to Helium. Note that this has also happened with a few Hydrogen upgraded to Helium routers as well. Now I have an issue where they randomly stop accepting any shell input. Everything else runs perfectly and I can leave the router running for weeks in this fashion. SSH still responds but as soon as I enter the username and password it will just sit there. Same response from console as well. To get back to a working console I have to power cycle the VM. It seems to happen after a random amount of time as well so sometimes it just takes minutes other times it runs fines for days. I’ve exhausted my Linux troubleshooting capabilities on this as I can’t find anything being referenced in /var/messages. I haven’t tried making any fresh builds of the VMs yet to see if it is just an issue with upgrading. I can confirm never running into this issue with Hydrogen though.
Any ideas to set me on the right track for getting this issue squashed?
I presume you have checked that your vyos box is configured to talk to a dns server (set system name-server some.ip.add.ress) so that it can do reverse dns resolution on the incoming ssh connection. That might cause long delays, but it should eventually timeout and let you login - unless something is broken and we have infinite delay in that case.
You might check /home/vyos/.bashrc and /home/vyos/.profile against those files from a fresh install - do they differ?
Was this installed via “install system” or “install image”? If you used install system, then you can add logging and write files in /home/vyatta that will still be there after a reboot. There must be some path that will be preserved across install image reboots - possibly /opt/vyatta/etc/config/scripts
I would try instrumenting /etc/bash.bashrc and /home/vyos/.bashrc with lines like
Sorry, should have come back and updated this thread after figuring things out. The resolution was to disable the VMware tools quiesce on the Veeam backups that I was running. Most likely you’ll need to either disable quiesce in the backups you are using or just exclude the router from the backups entirely.
Also seeing this, using Actifio for backups however (same deal with a ‘consistent’ backup via the VMware API though).
Perhaps worth noting that when the SSH/Console login issue occurs SNMP from my Cacti graphing is also unable to pull stats, and the memory usage in VMWare shows my VyOS VMs fully using the memory assignment (usually below ~300MB).
I can reproduce this if anyone wants some debug info, is it worth raising bug as it seems to be introduced in Helium?
I have faced the problem on my vyatta server. I am unable to login on my vyatta server through CLI, and i’ll try to login with GUI. I am unsuccessful to login.
ERROR when i used ssh
ssh -l vyatta 192.##.##.##
vyatta@192.##.##.##'s password:
Last login: Sat May 16 15:23:13 2015 from ###.###.##.###
Linux vyatta 2.6.31-1-586-vyatta #1 SMP Fri Mar 19 12:15:52 PDT 2010 i686
Welcome to Vyatta.
This system is open-source software. The exact distribution terms for
each module comprising the full system are described in the individual
files in /usr/share/doc/*/copyright.
Connection to 192.##.##.## closed.
ERROR when i login directly on server
Welcome to Vyatta - vyatta tty1
vyatta login: vyatta
Password:
Last login: Sat May 16 15:23:13 2015 from ###.###.##.### on pts/1
Linux vyatta 2.6.31-1-586-vyatta #1 SMP Fri Mar 19 12:15:52 PDT 2010 i686
Welcome to Vyatta.
This system is open-source software. The exact distribution terms for
each module comprising the full system are described in the individual
files in /usr/share/doc/*/copyright.
my suggestion is to boot to single user mode, copy off your config and upgrade to a newer version.
OR
stick your drive into any linux computer, and pull the config off, then upgrade to something current.
Please note that this is the VyOS forums, and not Vyatta. The 2 are very similar when running 6.x versions of Vyatta and the current builds of VyOS, but based on your version, you are likely to not be able to get much help here. Many changes happened in the 5.x and 6.x builds of Vyatta