Extremely slow command line response, mysterious "b" processes and SSHD messages - have I been p0wned?


#1

Hi! Today I logged into the vyos VM with SSH, and the system seems very slow. For example, logging into the router via SSH, I see the welcome message, and then it takes 1m10s to show me the prompt.
Then, I entered the ‘conf’ command, and it took 1m43s to enter into the configuration mode.

The ESXi monitor is telling me that the VM is consuming 100% of the vCPU (2.9GHz) and 100% of the RAM (1GB).

The configuration is extremely basic, and the router is only do basic static routes between the physical ESXi interfaces and the virtual switches.

On the ESXi console for the router, I’m seeing the following:

INIT: Id "TO" Respawning too fast: disabled for 5 minutes

I checked the logs too:

~$ show log
<datetime> Getty[xxxx]: ttyS0: tcgetattr: Input/Output error

The last line is repeated about 3 times per minute.

I’m also seeing a lot of login attempts from the internet, which is bothersome but not unusual I guess.

However, what I’m really worried about are lines like this:

<datetime> sshd[xxxx]: error: connect_to <random hostnames> 443: failed
<datetime> sshd[xxxx]: error: connect_to <random hostnames>: unknown host (name or service not known)

Where ‘random hostnames’ are things like this (I’ve deliberately added spaces to some FQDN’s below so as to not create links in the forum):

  • wpad
  • hakgaay
  • ysccxeafwceb
  • qpancgjwa
  • s7.addthis .com
  • 1e6795fd-f6e1-4297-a225-c50eb2a46569.browser.ip-score .com
  • sb.scorecardresearch .com

Running ps -e shows thousands of processes name ‘b’, many of them <defunct>

Have I been hacked? Is our router pegged its CPU because of some rogue process spamming the internet?


#2

Looks like indeed your vm was hacked
that random process names usually either send spam or ddosing/scanning networks


#3

best will be save your config
make new vm and load config there
Please firewall ssh or configure key based auth


#4

can you export and upload vm somewhere?


#5

I’d be happy to, could you recommend a place to do so? I assume you’d like to let g in and investigate?


#6

Hey,
that is correct.
you should be able to export it to ova (if that is vmware)
can upload to google drive or onedrive
you should remove any sensitive info before export
Thanks!


#7

Any news?
It would be great to understand how this router was hacked - weak password or whatever.


#8

you still have that vm?


#9

Sorry for the delay - I suspect it was the fact that the default password for the vyos user was left in place is what caused the breach. I’ve already talked to the tech in question, and they won’t be doing that again…

Unless there is a strong desire to see this image, I will not bother with the export.

Thanks for your quick responses though - highly appreciated and it adds to the appeal of the VyOS system in general.


#10

That make sense, some time ago we noticed that vyos now on brute force dictionaries, we treat this as fame :slight_smile:
i think we must add password change on install or maybe some warnings


#11

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.