Yet another Denial of Service (DoS) attack

In the past few days, we have seen another Denial of Service attack on a client's site.

The symptoms were a complete outage of the server, with very high CPU usage, and high load average (over 900 in some cases!).

Upon investigating, we found that this is caused by the following hits:

75.145.153.237 - - [22/Jul/2012:19:55:07 -0400] "POST / HTTP/1.1" 500 539 "-" "-"
75.145.153.237 - - [22/Jul/2012:19:55:07 -0400] "POST / HTTP/1.1" 500 539 "-" "-"
75.145.153.237 - - [22/Jul/2012:19:55:06 -0400] "POST / HTTP/1.1" 500 539 "-" "-"
75.145.153.237 - - [22/Jul/2012:19:55:07 -0400] "POST / HTTP/1.1" 500 539 "-" "-"
75.145.153.237 - - [22/Jul/2012:19:55:06 -0400] "POST / HTTP/1.1" 500 539 "-" "-"
75.145.153.237 - - [22/Jul/2012:19:55:07 -0400] "POST / HTTP/1.1" 500 539 "-" "-"

So, a script/bot was used to post data to the home page of the site, and that caused a service unavailable error to be returned.

All the IP addresses belonged to University of Illinois at Urbana-Champagne (UIUC), or to Comcast customers also in Illinois.

For the first and second incident, we blocked the IP address, or entire subnets. However we soon realized that this is a futile effort, since other IP addresses will be used.

We then devised a plan to prevent the POST request from reaching PHP altogether. This was done by adding the following to Drupal's .htaccess. Basically, it returns an access denied right from Apache if the conditions were met. The conditions are: empty referer, empty user agent, POST request to the home page.

# Modification for dealing with botnet DDoS via high CPU utilization
# by overwhelming PHP with POST data
#
# Referer is empty
RewriteCond %{HTTP_REFERER} ^$
# User agent is empty
RewriteCond %{HTTP_USER_AGENT} ^$
# The request is for the home page
RewriteCond %{REQUEST_URI} ^/$
# It is a POST request
RewriteCond %{REQUEST_METHOD} POST
# Forbid the request
RewriteRule ^(.*)$ - [F,L]

After implementing the above fix, the hits were successfully deflected, with no ill effect on the site.

67.177.109.10 - - [23/Jul/2012:16:31:23 -0400] "POST / HTTP/1.1" 403 202 "-" "-"
67.177.109.10 - - [23/Jul/2012:16:23:58 -0400] "POST / HTTP/1.1" 403 202 "-" "-"
67.177.109.10 - - [23/Jul/2012:16:31:21 -0400] "POST / HTTP/1.1" 403 202 "-" "-"
67.177.109.10 - - [23/Jul/2012:16:23:14 -0400] "POST / HTTP/1.1" 403 202 "-" "-"
67.177.109.10 - - [23/Jul/2012:16:23:29 -0400] "POST / HTTP/1.1" 403 202 "-" "-"
67.177.109.10 - - [23/Jul/2012:16:26:58 -0400] "POST / HTTP/1.1" 403 202 "-" "-"

Obviously, this is not the only protection for this type of attack. Other ways include installing Suhosin, and fiddling with Drupal's .htaccess as well, as described in PSA-2012-001.