防御僵尸networking – 如何分析apache日志?

大约24小时前,我的网站发起了一场DDOS攻击。 Apache日志看起来像这样:

190.56.92.50 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3a) Gecko/20030105 Phoenix/0.5" 79.162.132.75 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 SeaMonkey/1.0.4" 178.95.250.125 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Microsoft Internet Explorer/4.0b1 (Windows 95)" 41.224.117.207 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.1 (compatible; MSIE 5.0; Symbian OS; Nokia 6600;452) Opera 6.20 [ru]" 95.133.12.150 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1)" 78.139.156.77 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Opera/9.60 (Windows NT 5.1; U; en) Presto/2.1.1" 91.86.86.111 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 5.0; SunOS 5.9 sun4u; X11)" 46.73.152.227 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" 178.212.247.179 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; ; Linux armv5tejl; U) Opera 8.02 [en_US] Maemo browser 0.4.31 N770/SU-18" 189.243.119.252 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Symbian OS; Nokia 6600/5.27.0; 6329) Opera 8.00 [ru]" 187.159.55.110 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/2.0 (compatible; MSIE 3.01; Windows 98)" 79.162.132.75 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Opera/9.0 (Windows NT 5.1; U; en)" 46.118.8.181 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0)" 213.110.96.153 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Symbian OS; Nokia 6630/4.03.38; 6937) Opera 8.50 [es]" 46.201.147.160 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.50" 201.230.239.84 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1)" 201.230.239.84 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Opera/9.01 (X11; Linux i686; U; en)" 31.223.223.66 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5" 

正如你所看到的,所有的请求都会使用相同的url: browse.php ,当然,还有来自不同IP和用户代理的请求。 我拒绝使用.htaccess访问browse.php ,所以攻击不会通过重载Web服务器来影响其他页面/站点。

现在我有最后24小时的apache访问日志,大小为1.4 GB。

我怎么能根据日志得到一个IP列表,按他们访问browse.php的次数sorting,得到了403 Forbidden错误?

试试看: cat logfile | grep -P 'browse\.php\S* HTTP\/1\.\d"\s403' | cut -f1 -d" " | sort | uniq -c | sort -n cat logfile | grep -P 'browse\.php\S* HTTP\/1\.\d"\s403' | cut -f1 -d" " | sort | uniq -c | sort -n

尝试安装fail2ban并将其configuration为apache。 Fail2ban实际上阻止了多个机器人的并行请求,并且可以很容易地configuration为阻止基于特定正则expression式的请求。

一旦我试图手动阻止他们,但它是一个巨大的痛苦,因为他们是数百万在那里,你会厌倦手动阻止他们通过自制的脚本。