One Client Site Hacked

[belindaj]

Disclaimers: (so we can hopefully get right to the meat of the situation and skip the things that I should/do already know and have in place.

1. I do not have anonymous FTP enabled on ANY of my sites/servers.

2. All my patches ARE up to date ( always)

3. My "resellers" or designers - do not have the permissions ability to toggle anonymous FTP on or off.

4. My administrator password is changed regularly and is a "strong" password.

Now to the request for help.

One of the sites on my Windows 2000 server has recently been exploited - to the tune of 8.3 GB of used space. When I started receiving bandwidth notices last week, and notice this morning that I had ZERO space to upload files to a different site on this server, I began looking into the cause. We'll leave off the little part about how Interland's support technician could have been a bit more customer-centric by answering a STUPID SIMPLE QUESTION last week instead of noting the ticket with "If you require a more in-depth assessment to determine the cause in bandwidth usage, please reopen this ticket authorizing charges of $45 per half hour for this service..." Oh yeah, I said we were going to leave that part off.... (yes Chicken - I STILL have one server over there - )

But I digress.

Anyway - I found the rogue files/folders/directories and they go many, many levels deep - with non-standard file names of course. As I navigate all the way to the bottom of some of them - it's evident that they were uploading DVDs to this domain (not the owner of the domain - the hacker(s)). It is ALL contained within this single domain - no other intrusion anywhere on my server - which leads me to believe that this particular designer's password was cracked - not the server password.

Nevertheless - I immediately changed the server password, and the password to the domain itself, and notified the domain owner that his own machine may have been compromised since it appears to be confined to only his website account. He has no access to the site now until I complete the cleanup.

The cleanup is not working. I cannot delete these files/folders - either through the file directory structure itself, through the command line DOS (this excerpt from one website that I've been working through various fixes: "I would get about four subdirectories down and it wouldn't let me cd to the next directory down (in this case, named "con"), so I couldn't get to the bottom of the tree to delete from there up." - is my exact same problem), or using a program called Tritafile which worked for the person whose comment I excerpted, but is not working on my server. FYI - the link to this particular googled set of solutions is http://www.msfn.org/board/index.php?showtopic=8509&st=10

Any suggestions from someone who has been through this?

This is a Windows 2000 server.

[belindaj]

I've managed to get it down to 3.9 GB from the 8.3 GB earlier - but there's a mess of directories & subdirectories that just will not budge.

Navigating down the tree from within the DOS prompt doesn't help because I can't get all the way to the bottom to work backwards.

Much appreciation for suggestions to try.

[belindaj]

FYI - I have now tried several of the Microsoft KB suggestions - including:

DEL \\.\C:\sharename\reseller\domainuser\domain.com\ww w\5444948.137, Are you sure (Y/N)? y

And it simply returns the prompt again. The directory is not deleted.

I have also tried

use dir /X to get the 8.3 folder name (doesn't work)
then RD /S and path\8.3 foldername (tried using one of the dir names that is not more than 8 characters - also didn't work on that dir)

I did bounce the server, in the hopes that it would jog the stuck folders - then tried the Tritafile program again - and THOUGHT it worked - until I realized that it didn't actually shred the files/directory - it reappeared with a new name with all the subdirectories intact under the new name.

[belindaj]

Have tried stopping IIS service (duh) - THEN deleting - still won't go away. Removed the site in Hosting Controller. Won't go away, although now the DNS entries are gone (not a biggie - my thought process on this one was to remove the site altogether and recreate it - on this server or one of my other servers - from scratch)

[alwaysconfused]

I havent a clue about webhosting and servers but couldnt you shutdown the server and make it come back on again and change everything and put everything back on it? like the pw and the users and all that? -Confused

Yeah you probably cant but since I dont know anything about this, its the only recommendation I have!

[Cow]

This problem sounds familiar. Would you mind me helping you? I don't charge anything, I just hate to see other servers being compromised.

Also, give us an update on the problem. This sounds like something we've seen before.

[belindaj]

Everything I've done (successfully and unsuccessfully) is logged in the previous responses. I posted as I tried fixes - hoping someone would see something that was maybe not as obvious to me.

I'd certainly welcome additional suggestions. I did post over at WHT as well since no one was replying here - but the responses there were not as helpful as I was hoping for (although well-meaning).

I'll post a screenshot of an example of the directory tree structure that I am unable to remove. I'd welcome additional suggestions.

At present - I've removed the DNS from the server, we changed nameservers & copied the static, uncompromised portions of the site entirely to a different server. The client is scanning his home machine for possible trojans, spyware & viruses. I've changed administrator password on the server box, and changed password on the individual domain access. An associate of mine googled the web yesterday for solutions and also checked a known "compromised IP" list that she came across and my server IP is not listed (on that one anyway).

I think I'm as locked down as I can be at present - and I just want to get the rest of these files removed asap. This is not a heavily used server, but the few sites on it are mission critical for their owners of course. As I said earlier in a post - I did regain about 5 GB of the space so the other sites are again accessible for editing.

Thanks James

Belinda

[belindaj]

oops forgot to post the screenshot.

[Cow]

Yep We had a similar problem. I'm just trying to think how we rectified it.

[Chicken]

Originally posted by belindaj:

I did bounce the server, in the hopes that it would jog the stuck folders - then tried the Tritafile program again - and THOUGHT it worked - until I realized that it didn't actually shred the files/directory - it reappeared with a new name with all the subdirectories intact under the new name.
So even when you remove the directories, they come back?

From command line...

C:>rmdir/s directory

I didn't see that you mentioned that one, but maybe you've tried it. Also not sure what happens if you just removed the entire site and then rebuild it (sans the dvd files of course) ???

[belindaj]

So even when you remove the directories, they come back?

From command line...

C:>rmdir/s directory

Yes - rmdir didn't work at all, and after bouncing the server and trying the Tritafile script again - the directory went away - only to be replaced by a new one with a new name - with the same tree underneath it.

Tried removing the entire site - it won't remove. When I couldn't delete it from inside IIS or Explorer itself - I tried removing it from Hosting Controller - and it did remove it from Hosting Controller - (all the DNS entries anyway) - but the site itelf (files & www directory) still stayed put on the actual drive.

[Chicken]

That really doesn't sound good. The only time I've personally experienced anything remotely like that was with an .exe that was on my computer and similar to what you mentioned, it either couldn't be removed, or when I finally did get rid of it, seconds later a new .exe appeared with a random name.

I ended up wiping the machine.

There must be some info on the net about this, as I'm sure it's happened to someone else before. Will try help look later.

Although it appears that just the site is compromised, it sure seems like the server may now by compromised (deeper problems than just the site itself).

[belindaj]

When the directory renames itself - it is still completely confined to the same domain location - nowhere else on the server - no other sites - no other drives (my webspace is not on the C drive) - and there has been no unusual activity since my work on Monday when I found the intrusion, remediated as best I could, and took preventative steps to further lock down the server.

I am aware that there may be things I am missing - which is why I hope someone here can point out some other concerns.

For now I'm in a holding pattern. I'm very thankful for any advice thrown my way.

[Chrysalis]

doesnt deltree command work?

deltree is a standard dos command that will wipe all files in a directory including but not limited to subdirs, hidden files, system files , read only files.

[x-2o]

What do the files hold ?