Three security problems on my server. How do I fix it?

[nman14]

Ok, I have had three problems in the past. The first was I had someone sending spam emails from my server and the provider shut my server down. ( not even giving me time to make backups) The second problem I have had was a hacker getting into my server and starting to delete many accounts. I rebooted the server and changed the pass before he could get to them all. And the last problem I sometimes have is clients using too much of my servers resources. Now lets get to the questions:

1: How do I stop people from sending spam emails? Is there a way to set how many emails can be sent out in an hour? If so how do I set it? (Cpanel/whm Linux fedora)

2: I had a firewall installed when the hacker got into my server but not BFD. Could this have been the problem? I also had somewhat of a week password. (xxxxxx##) What can I do to prevent this from happening again?

3: How do I limit the amount of recourses an account can use?

any help would be great. Thanks!

[Bloory]

Re; No2 - I would switch to a key pair which is far more secure, possibly remove root login. If the server has been compromised contact someone like www.rack911.com or www.configserver.com as they will perform a full audit.

Both can also advise about No 1 - keeping scripts up to date, running phpsuexec or suphp and a mail header patch so tracing any sent mail after the fact.

Resource usage is managed through WHM (defining packages and assigning them to accounts or using the reseller center for resellers).

[ganesh.rao]

Resource usage is managed through WHM (defining packages and assigning them to accounts or using the reseller center for resellers).OP meant CPU and RAM, right?

I am not sure if this is possible at all.

[nman14]

Yes, I was asking about limiting Ram and CPU usage for the users. Thanks for your responses so far.

[EWH-Chris]

Chirpy made a nice firewall addon for cPanel servers and offers it free of charge- http://www.configserver.com/cp/csf.html
It also has settings to help with spammers, such as limited PHP script emails. I have mine set to 100, so when they hit 101, it disables their site and emails me. Because of this, I let my clients know to use SMTP auth for their forums and such and help them set it up if needed, but it has cut down spamming from my servers so much!

[Dimension4]

Re; No2 - I would switch to a key pair which is far more secure, possibly remove root login. If the server has been compromised contact someone like www.rack911.com or www.configserver.com as they will perform a full audit.

Both can also advise about No 1 - keeping scripts up to date, running phpsuexec or suphp and a mail header patch so tracing any sent mail after the fact.

Resource usage is managed through WHM (defining packages and assigning them to accounts or using the reseller center for resellers).

We used to stray away from phpsuexec on our shared servers, but made the switch and I wish we had done it sooner. It prevents a lot of problems from hackers, although keeping scripts up to date is still very imortant.