High CPU load caused by kswapd0

Hi all,

I notice a high cpu load after pushing data to threw the DMVPN cloud. Am not sure if it has anything to do with the VPN but the CPU load is not dropping even when hardly any packets are being processed.

kswapd0 is asking all the cpu resources and i cant seem to figure out why this is.
Is this a bug or whats going wrong?

Gr,

As far as I know, kswapd0 overuses cpu resources, which means that virtual memory replacement occurs frequently. There are two common solutions:

  1. Increase the memory capacity and set the following system parameters:
vm.swappiness=0

Setting this parameter will require the Linux system to use physical memory as much as possible, and at the same time cancel the swap partition to improve system performance

  1. In the case of severe system memory shortage, the linux virtual memory management mechanism has to use disk resources, which will cause frequent page replacement. If the memory has a certain capacity, you can consider increasing the vm.swappiness parameter to improve CPU resources as much as possible Utilization rate, while using more efficient processors.You can consider allocating swap resources for your system. Try to relieve memory strain.Increasing swap storage is also a general way to supplement physical memory

If you cannot solve this problem temporarily, please pay attention to whether the occupancy rate is high for a long time. If it only appears temporarily and the system load is within the normal range, you can consider optimization

You can also consider submitting a report at https://phabricator.vyos.net, providing detailed information so that relevant personnel in the vyos community can confirm whether it is in a normal situation. Usually, in this situation, your system resources can be given priority Already severely insufficient

Hello @Arpanet69, who had access to this router, it looks like kswapd0 is just a malicious script
Try to find and rename it, then just kill this proucess

sudo find / -name kswapd0

Which VyOS version used? Provide an output of the command show version
Where you get this VyOS version?

1 Like

@Arpanet69 will be interesting to research this situation. When you find this script file, check the file creation date time and try to find information in the log about access to this router.

Well, I did not pay attention to the path of the process, it seems to be a malicious script, as @Dmitry said

The malicious script seems to be disguised as a system process. The process in my kubuntu is as follows:

Wel i think am the only one who has access to this router.
I will try to force the issue tomorrow since the reboot solved the issue. It seems for now that this process starts eating cpu when sending large files over the DMVPN tunnel. I will push a large file and collect loggings. Any advise which loggings would be usefull to gather?

Version: VyOS 1.3-rolling-202010220152
Release Train: equuleus

Built by: autobuild@vyos.net
Built on: Thu 22 Oct 2020 01:52 UTC
Build UUID: ad4783ef-9e01-4a2a-80a4-351a87624528
Build Commit ID:

Software downloaded from the Vyos website not from any other websites or torrents.

Greetings

Please note that the kswapd0 path is forged

Hello @Arpanet69
I propose to collect all journalctl output to a file

sudo journalctl > /tmp/log

And also will be helpful to see an output

show tech-support 

Did you install custom packages?

Sorry for my late response guys i was pretty sick. Meanwhile i got some notifications from the ISP that there were brute force attacks been running form my router… What the hell.

Ive upgraded to the latest rolling but this is not fixing the issue.

@ACS0001-CE1:/$ sudo find / -name kswapd0
/boot/rw/home/vyos/.configrc/a/kswapd0
/home/vyos/.configrc/a/kswapd0
/tmp/.X25-unix/.rsync/a/kswapd0
/usr/lib/live/mount/persistence/boot/1.3-rolling-202012171749/rw/home/vyos/.configrc/a/kswapd0
/usr/lib/live/mount/persistence/boot/1.3-rolling-202010220152/rw/home/vyos/.configrc/a/kswapd0
@ACS0001-CE1:/$ cd /tmp
@ACS0001-CE1:/tmp$ ls
log                                                                      systemd-private-4c8fbec897da4685a30edac80fe42b1f-ntp.service-QUoDbh
systemd-private-4c8fbec897da4685a30edac80fe42b1f-haveged.service-aDohOt  uacctd.pipe
systemd-private-4c8fbec897da4685a30edac80fe42b1f-lldpd.service-WsKW9e    vyos-config-status
@ACS0001-CE1:/tmp$ cd /tmp/.X25-unix/
@ACS0001-CE1:/tmp/.X25-unix$ ls
dota3.tar.gz
@ACS0001-CE1:/tmp/.X25-unix$ cd ~/.ssh/
@ACS0001-CE1:~/.ssh$ ls
authorized_keys
@ACS0001-CE1:~/.ssh$ nano authorized_keys 
@ACS0001-CE1:~/.ssh$ cd /tmp/.X25-unix/
@ACS0001-CE1:/tmp/.X25-unix$ ls
dota3.tar.gz
@ACS0001-CE1:/tmp/.X25-unix$ 

dota3.tar.gz?? I dont dont think that should be there… :slight_smile:

Anybody familiar with this? How can i safely disable this? Do you want me to pull some interesting info before i completely erase the box and reinstall.

Greetings

Alright its fixed now.
some script was running in the background i deleted it and killed the process.

Thanks for noticing this was a malicous process.

Arpanet

Could this malicious code came from the vyos site when i did an upgrade since there is this kswapd0 file in the path of the os image?

/usr/lib/live/mount/persistence/boot/1.3-rolling-202010220152/rw/home/vyos/.configrc/a/kswapd0

Hello @Arpanet69, I’m sure that this malicious code not related to updates.
It will be interesting to look into this code and check date/time when this file added. Then check in VyOS log what happens in finding timestamp

Hello,

I also faced same issue today. My fault was, I kept the default user credential intact (vyos/vyos)

Thanks

Has anyone captured the file or script that is malicious?

@Arpanet69 did you expose your system to the internet with the default username and password at any point?