Dealing with a Hacker on Dreamhost
30 May 2009
Earlier this year, my Dreamhost webhosting account was hacked. I’ve been dealing with it for months, but I’m no programmer. The information provided by Dreamhost customer support, while helpful, has been far from sufficient to actually resolve the problem in a satisfactory way. That’s why I’m writing this blog post: to help others than might be in a similar situation.
How the Hacker Got In
I’m pretty sure the hacker got in through an old abandoned WordPress install that I had forgotten to delete. (It’s essential that you either keep all web apps up to date, or delete them. To do otherwise is to ask for trouble. Hackers will eventually discover the old installs with security vulnerabilities.)
After gaining access, the hacker uploaded a PHP backdoor script which allowed him to get back in easily and upload or edit any files he wants, even after I deleted the old WordPress installation that had the vulnerability. The backdoor script he used is called PHPspy, and is freely available on the internet. (Interestingly, it’s also Chinese.)
What the Hacker Did
The hacker was not overly malicious as hackers go. He was after precious Google Page Rank. He took a three-step approach to his spam injections:
1. Add a new directory full of “male-enhancing” spam content (hundreds of separate HTML files, optimized for all kinds of possible searches).
2. Add a harmless looking PHP file, like
css.php somewhere on your webspace where it seems to fit, like in a WordPress directory. This file contains a PHP eval base64 decode function. This allows the file to hide what it’s actually doing; it has to be decoded to be readable by humans. (There are sites which can decode base64 online for free.) If you decode that content, though, you’ll see that it’s including all the spam content added in step 1.
3. Edit your WordPress blog’s theme’s
header.php file to include the innocuous-looking file added in step 2. It is added in a way that is invisible to visitors, but highly prominent to the Googlebot.
The effect of all this is that (1) searches for all those “male-enhancing” drugs on your site reveal tons of hits, and (2) the pages it points to all go lead to some other site, which reaps the Google page rank benefits.
It has taken me quite some time to figure all this out. Dreamhost, so great in many ways, takes a very hands-off approach to security issues. I was given this advice by Dreamhost staff:
> After updating your software, it is imperative that you go through all files under all directories for the user which has been compromised and ensure that any files which have been written to or modified have been removed. It is common for ‘hackers’ that exploit web scripts to upload innocuously-named scripts which they can use to further compromise the site more easily, even after the initial vulnerability is closed — including scripts to send spam mail or execute arbitrary shell commands under your account via a simple web page interface. A helpful tip for finding files of this nature is to look for files or directories that have timestamps that occurred since you last modified your site, or that occurred around the time that the ‘hack’ took place; still it is best to examine all files as even a single missed file can allow the site to be re-compromised.
As you can see, this is good advice, and it fits my situation perfectly. It was the “date modified,” viewable by FTP, that enabled me to find the hacked/uploaded files and directories in the first place.
Unfortunately, for an account with web content as sprawling as mine (I host a number of other sites in addition to Sinosplice), manually going through every directory isn’t an option. Even if you were willing to do this (it would take many hours), you would have no guarantee that the hacker wasn’t accessing the site right behind you, re-uploading backdoor scripts in directories just after you’ve cleaned/cleared them.
So it sounds like the best solution is deleting everything and re-uploading. This, unfortunately, isn’t the easiest thing to do either when you have multiple copies of WordPress installed, each of which is tied to Dreamhost’s automatic one-click installer system. Ironically, it’s also the Dreamhost one-click installer system that adds to the problem. The Dreamhost one-click installer adds no less than 50 themes to each copy of WordPress it installs. That’s 50 extra potential hiding places for a hacker’s backdoor scripts, one of the reasons why a manual check is not an option. (You can delete the themes you don’t want, but it’s a slow process by FTP, and you have to do it again, on every WP install, every time you upgrade. Even if you do it through your shell account, some of the themes have permissions set in such a way that a simple delete won’t work.)
Solving the Problem
The first thing you have to do is shut out the hacker. Until you can do that, it’s pointless to spend much time cleaning everything up. The “hacker” may in fact be an automated script. It may never tire of replacing all the files you delete. Obviously, you must shut out the hacker first.
Dreamhost provides a page which helps you figure out how your site was compromised. In my case, I was able to confirm that neither my FTP nor my shell account had been compromised; it seemed to be a simple case of web access.
After determining that my hacker was using PHPspy to gain entry, I knew I had to find all instances of PHPspy on my webspace and remove them. To do this, you’re going to need shell access. You can enable it through the Dreamhost control panel. (If you have no idea what shell access is, you probably shouldn’t try to use it. It’s not user-friendly at all.)
Anyway, once you have shell access, you can start using the find command. Searching for filenames is pretty useless, since the hacker can name his files whatever he wants. You need to search for strings in files. By actually searching the contents of files, you can easily identify (1) spammy or unwanted content, (2) any files containing the very suspicious string
eval(base64_decode(, (3) the files including the files with the
eval(base64_decode( strings, and, most importantly, (4) backdoor scripts (if you know what they are). In my case, I knew all the “male-enhancing” keywords to search for, and since I had found a copy of PHPspy on my webspace (innocently named
log.php), I was able to open it up in a text editor and choose some unique strings in the script to search for.
After I identified all those files, it was simple to (1) delete the files that shouldn’t have been there (especially the PHPspy scripts!), and (2) re-upload the legitimate files to overwrite the ones that had been hacked to include the spammy stuff.
Finally, to monitor the changes made to files on my webspace, I set up a monitor script as described in step 3 of 3 on this forum post. I deleted the “Recent Logins and Associated Hosts” part because it was resulting in tons of useless data.
Again, I’m disappointed that Dreamhost couldn’t help more. I’ve been a loyal customer for years, and have referred quite a few new customers. It’s just not their policy to get directly involved, however.
The good news is that the non-urgent nature of my particular security situation gave me ample time to investigate, consult (big thanks to Brad and John Levermore for their help), and experiment. It’s my hope that this information can help others in similar situations.
A big chunk of my free time in the past few weeks has been taken up by this security issue. I’m certainly no expert, so any corrections or advice from those more knowledgeable are welcome! Hopefully this is the last you’ll hear of it, and I can go back to writing more Chinese-related stuff for Sinosplice.