Banning Bad Bots: A Short But Effective Script

Introduction

Lots of malicious machines are out there trying to find exploits on your web site. From the occasional armchair script kiddie looking for an unpached IIS to a full-scale botnet bent on finding and exploiting email scripts, chances are your site is being hit with far more useless–not to mention potentially dangerous–traffic than necessary.

Because you're a good programmer you're always on top of the latest patches for your OS, Web server, and apps; or if you're on a shared server you're smart enough to go with a company that does this for you, right? Right.

But surely there's an easy and fun way to stop all those bogus requests, if for no other reason than to slam the door shut on all those Web equivalents of door-to-door snake oil salesmen. If they're looking for exploits, who's to say they aren't also scraping your site for email addresses?

Overview

For those of you who want to cut to the chase, here's the general idea: for every URL that shouldn't ever get legitimate traffic, map it (e.g. via Apache's mod_rewrite) to a CGI that bans that IP address by adding a "deny from x.x.x.x" line to .htaccess.

In other words, the moment a host attempts to retrieve a dangerous URL your script adds the host's IP address to your site's list of banned hosts, thereby locking them out of retrieving anything on your site.

I'll give you Perl and PHP versions of this script, as well as some ideas for modifying it to make it more robust.

Perl

Here's a Perl script that does the job:

 

#!/usr/local/bin/perl -w

 

use Fcntl qw(:flock);

use CGI qw(:standard);

use IO::File;

 

my $fp = new IO::File();

 

$filename = '/path/to/.htaccess'; # CHANGE IF NECESSARY

if (open($fp, ">>$filename")) {

eval { flock($fp, LOCK_EX); };

print $fp "deny from $ENV{'REMOTE_HOST'}\n";

 

eval { flock($fp, LOCK_UN); };

close($fp);

}

 

print header(-status => "HTTP/1.1 403 Forbidden");

 

exit();

 

# EOF

PHP

Here's the PHP equivalent:

<?php

 

$filename = '/path/to/.htaccess'; // CHANGE IF NECESSARY

 

if ($fp = fopen($filename, 'a')) {

if (flock($fp, LOCK_EX)) {

$remote_addr = $_SERVER['REMOTE_ADDR'];

 

fwrite($fp, "deny from $remote_addr\n");

}

 

flock($fp, LOCK_UN);

 

fclose($fp);

}

 

header('HTTP/1.1 403 Forbidden');

 

echo "Forbidden!"; // Add any other HTML here

 

exit();

 

// EOF

 

?>

Customizing Your Script

I'm assuming you know enough Perl or PHP to properly configure the line in the script that contains the path to .htaccess on your server ("CHANGE IF NECESSARY"). You'll also need to ensure .htaccess is writable by your web server.

Linking to Your Script

Next, upload your script (let's call it denied.php) to your server. Theoretically you can place it in any directory, but I'll assume you've placed it in the root.

Now we need to link potentially malicious URLs to your script. One easy way is to use Apache's .htaccess:

RewriteEngine on

RewriteRule bbs/skins$ denied.php [L]

RewriteRule email.cgi$ denied.php [L]

#... add more rewrite rules here ...

In this case I've redirected requests for "bbs/skins" and "email.cgi" to denied.php. Obviously you'll want to customize this for your site.

Testing It Out

Test it out by trying to navigate to one of your banned URLs. For example, I would surf to:

http://www.my-site-here.com/bbs/skins

Of course, once you've successfully tested it by banning yourself you'll want to unban yourself. Just remove the "deny from x.x.x.x" line that was added to .htaccess.

Extras

Extra features you might want to add:

• Automatically unban hosts after a given amount of time. To do this you'd have to track when each host was added to .htaccess. You could store that info in a database, or as comments in .htaccess.

• Keep the connection open for a while. This is the equivalent of telling the telemarketer to hold on while you put the phone down and have a cup of coffee before hanging up on them. While it probably won't have much effect on the performance of the malicious host (and may even consume too many of your server's resources if you get too many requests), it's something to keep in mind.

• Don't deny access; instead, redirect the malicious host to a more "interesting" web site. In this case, "interesting" is left to your imagination. A porn site? A government site? A hacker site? You decide. While many of the hosts will simply ignore the redirect, some might be stupid enough to try it.

Potential Dangers

The real pitfalls here are that you may end up banning traffic you don't want to ban:

• Mistakenly banning legitimate traffic. If any of the banned URLs are listed publicly, or not otherwise kept away from legitimate Web surfers (individuals and/or bots), you may end up banning traffic you don't want to ban (e.g. Google).

• Mistakenly banning everybody on a proxy. If somebody using a popular proxy (AOL, anyone?) accesses a banned URL they'll end up banning your site for everybody on that proxy since they all appear to your site as coming from the same IP address.

Conclusion

Banning malicious hosts from your site is not only practical, but also it's lots of fun. Now all we need to do is find out how to profit from it.

===END===