nginx，独立的机器人访问日志和人类访问日志

我试图分开机器人访问日志和人类访问日志，所以我使用下面的configuration：

http { .... map $http_user_agent $ifbot { default 0; "~*rogerbot" 3; "~*ChinasoSpider" 3; "~*Yahoo" 1; "~*Bot" 1; "~*Spider" 1; "~*archive" 1; "~*search" 1; "~*Yahoo" 1; "~Mediapartners-Google" 1; "~*bingbot" 1; "~*YandexBot" 1; "~*Feedly" 2; "~*Superfeedr" 2; "~*QuiteRSS" 2; "~*g2reader" 2; "~*Digg" 2; "~*trendiction" 3; "~*AhrefsBot" 3; "~*curl" 3; "~*Ruby" 3; "~*Player" 3; "~*Go\ http\ package" 3; "~*Lynx" 3; "~*Sleuth" 3; "~*Python" 3; "~*Wget" 3; "~*perl" 3; "~*httrack" 3; "~*JikeSpider" 3; "~*PHP" 3; "~*WebIndex" 3; "~*magpie-crawler" 3; "~*JUC" 3; "~*Scrapy" 3; "~*libfetch" 3; "~*WinHTTrack" 3; "~*htmlparser" 3; "~*urllib" 3; "~*Zeus" 3; "~*scan" 3; "~*Indy\ Library" 3; "~*libwww-perl" 3; "~*GetRight" 3; "~*GetWeb!" 3; "~*Go!Zilla" 3; "~*Go-Ahead-Got-It" 3; "~*Download\ Demon" 3; "~*TurnitinBot" 3; "~*WebscanSpider" 3; "~*WebBench" 3; "~*YisouSpider" 3; "~*check_http" 3; "~*webmeup-crawler" 3; "~*omgili" 3; "~*blah" 3; "~*fountainfo" 3; "~*MicroMessenger" 3; "~*QQDownload" 3; "~*shoulu.jike.com" 3; "~*omgilibot" 3; "~*pyspider" 3; } .... } 

而在服务器部分，我正在使用：

  if ($ifbot = "1") { set $spiderbot 1; } if ($ifbot = "2") { set $rssbot 1; } if ($ifbot = "3") { return 403; access_log /web/log/badbot.log main; } access_log /web/log/location_access.log main; access_log /web/log/spider_access.log main if=$spiderbot; access_log /web/log/rssbot_access.log main if=$rssbot; 

但是似乎nginx会将一些机器人日志写入到location_access.log和spider_access.log中。

我怎样才能分离机器人的日志？

• 使用apache日志测量和可视化Web应用程序的性能？
• date的grep范围 – apache日志
• Unix / Linux简单日志parsing器（自从，直到）
• macports在哪里login它的stdout？
• 我应该如何处理日志，然后才能大？

另一个问题是一些机器人日志不会写入到spider_access.log中，而是存在于location_access.log中。 看来我的地图不工作。 当我定义“地图”时有什么不对吗？

 map $http_user_agent $bot { default ""; "~*Googlebot" "yes"; "~*MJ12bot" "yes"; # Add as many as desired } map $bot $no_bot { default "no"; "yes" ""; } 

 access_log /var/log/regular_access.log main if=$no_bot; access_log /var/log/bots_access.log main if=$bot; 

 map $http_user_agent $bot_header { default ""; "~*Googlebot" "bots_"; "~*MJ12bot" "bots_"; # Add as many as desired } map $server_name $log_filename { default "unknown"; "site1....." "site1_***.log"; "site2....." "site2_***.log"; } 

 server { # simple reverse-proxy... listen 37........:80; server_name dev.****.net; access_log /var/log/nginx/access/$bot_header$log_filename main; # pass all requests location / { # There, your config } }