Apache被某些请求“堵塞”了

我有一个非常奇怪的间歇性问题。 它发生在运行CentOS和Apache / 2.2.8的共享托pipe服务器上,主要以10-30个请求/秒的速率为基于PHP的网站提供服务。 3.5 GB内存,内核2.6.9-67.0.7ELsmp,以防万一。

会发生什么事情,大约每周一次(但看似随机没有关联),Apache将“堵塞”处理请求只有一个域,通常大部分的重新testing都是为同一个url。 我知道问题的发生是因为负载通过屋顶。 (负载达到50或100,如果我不停止加载命中200!)只修复我发现是: killall -9 httpd; /etc/init.d/httpd restart killall -9 httpd; /etc/init.d/httpd restart

如果在发生这种情况时检查/服务器状态,则输出类似于以下内容。 请注意,这并不总是发生在这个网站,但约75%的时间,这个网站的罪魁祸首: 编辑 :这是现在发生在一个完全不同的服务器,所以它不是特定于给定的脚本。

更新:我们已经把这台服务器上的所有站点都移到了一个全新的硬件虚拟机上。 更快的磁盘,更多的内存。 问题依然存在,实际上更糟糕! 有任何想法吗? 我很乐意提供一个正确的答案巨大的赏金,但我不愿意提供一个,因为这个问题已经走了多年,没有正确的答案在眼前。

 Apache Server Status for localhost Server Version: Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.7a DAV/2 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 PHP/5.2.6 mod_perl/2.0.4 Perl/v5.8.8 Server Built: Jun 7 2008 09:35:37 Current Time: Friday, 17-Jul-2009 10:15:40 EDT Restart Time: Friday, 03-Jul-2009 12:36:59 EDT Parent Server Generation: 3 Server uptime: 13 days 21 hours 38 minutes 40 seconds Total accesses: 10343186 - Total Traffic: 111.6 GB CPU Usage: u1710.63 s2129.19 cu.41 cs0 - .32% CPU load 8.61 requests/sec - 97.4 kB/second - 11.3 kB/request 250 requests currently being processed, 0 idle workers WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWWWWWWWWWWWWWW...... Scoreboard Key: "_" Waiting for Connection, "S" Starting up, "R" Reading Request, "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup, "C" Closing connection, "L" Logging, "G" Gracefully finishing, "I" Idle cleanup of worker, "." Open slot with no current process Srv PID Acc M CPU SS Req Conn Child Slot Client VHost Request 0-3 3282 0/52/316180 W 1.47 41802 0 0.0 0.54 3849.21 209.249.53.104 zachdotsey.com GET /wordpress/2007/08/18/the-body-exhibition/ HTTP/1.0 1-3 30061 0/191/313974 W 34.46 44051 0 0.0 2.62 3400.19 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 2-3 2720 1/23/307812 W 0.13 42298 0 0.5 0.01 3479.29 68.118.8.157 zachdotsey.com GET /wordpress/ HTTP/1.1 3-3 1733 0/382/306392 W 3.59 41744 0 0.0 3.67 3383.63 75.101.200.204 zachdotsey.com GET /wordpress/2007/08/18/the-body-exhibition/ HTTP/1.0 4-3 3141 0/132/305071 W 3.52 41776 0 0.0 2.20 3335.79 209.249.53.236 zachdotsey.com GET /wordpress/2007/08/18/the-body-exhibition/ HTTP/1.0 5-3 31819 0/73/300070 W 0.79 43932 0 0.0 2.05 3349.65 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 6-3 6799 0/13/294060 W 0.16 40354 0 0.0 0.25 3261.88 76.23.230.184 zachdotsey.com GET /wordpress/?p=111 HTTP/1.1 7-3 4334 0/92/289873 W 0.42 40277 0 0.0 0.36 3211.94 72.233.56.144 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 8-3 18326 0/11/294687 W 0.14 32476 0 0.0 0.06 3217.41 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 9-3 2451 0/56/291730 W 0.23 42381 0 0.0 0.32 3113.19 68.118.8.157 zachdotsey.com GET /wordpress/index.php HTTP/1.1 10-3 31211 0/59/280899 W 6.04 44051 0 0.0 0.77 3128.00 72.52.238.128 zachdotsey.com POST /wordpress/wp-cron.php?doing_wp_cron HTTP/1.0 11-3 3823 0/4/277229 W 0.17 41746 0 0.0 0.04 3014.66 68.118.8.157 zachdotsey.com GET /wordpress/index.php HTTP/1.1 12-3 4473 0/9/277074 W 0.08 38453 0 0.0 0.00 3313.94 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 13-3 4242 1/14/272166 W 0.10 41659 0 0.5 0.05 2977.05 68.118.8.157 zachdotsey.com GET /wordpress/ HTTP/1.1 14-3 2323 0/234/269513 W 0.84 42136 0 0.0 1.02 2946.48 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 15-3 32051 0/6/257740 W 0.08 43872 0 0.0 0.58 2866.34 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 16-3 16986 0/271/259463 W 18.13 16761 0 0.0 4.12 2896.05 74.6.18.233 zachdotsey.com GET /wordpress/tag/xbox/ HTTP/1.0 17-3 2297 1/8/252501 W 0.54 42502 0 0.5 0.18 2797.73 68.118.8.157 zachdotsey.com GET /wordpress/ HTTP/1.1 18-3 9476 1/145/247116 W 0.61 40095 0 0.4 1.53 2703.11 64.237.119.18 zachdotsey.com GET /wordpress/ HTTP/1.1 19-3 2494 0/214/244830 W 3.16 42017 0 0.0 2.40 2618.41 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 20-3 5526 0/118/238426 W 3.50 37656 0 0.0 1.73 2544.06 68.10.212.178 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 21-3 31901 0/28/228757 W 0.14 43992 0 0.0 0.04 2522.36 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 22-3 12398 0/25/216141 W 0.40 39998 0 0.0 4.72 2398.82 38.117.108.131 zachdotsey.com GET /wordpress/ HTTP/1.1 23-3 2300 0/187/222141 W 0.68 42315 0 0.0 0.67 2503.15 65.184.161.115 zachdotsey.com GET /wordpress/ HTTP/1.1 24-3 31969 0/25/211848 W 0.26 43810 0 0.0 0.38 2326.01 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 25-3 31582 0/25/198495 W 0.17 69010 0 0.0 0.25 2139.13 70.63.195.121 myvenue.venuecom.com POST /fastmail/ HTTP/1.1 26-3 28115 1/1/197089 W 0.04 40155 0 0.4 0.00 2126.75 64.237.119.18 zachdotsey.com GET /wordpress/ HTTP/1.1 27-3 12988 1/42/186898 W 0.55 35055 0 2.1 0.36 2037.54 71.193.156.250 zachdotsey.com GET /wordpress/ HTTP/1.1 28-3 3780 1/26/181659 W 0.34 41662 0 0.5 0.05 1918.00 68.118.8.157 zachdotsey.com GET /wordpress/ HTTP/1.1 29-3 1002 0/22/169331 W 0.18 39885 0 0.0 0.03 1888.59 64.40.121.190 zachdotsey.com GET /robots.txt HTTP/1.0 30-3 4493 0/6/163101 W 0.06 41465 0 0.0 0.18 1732.58 209.131.41.49 zachdotsey.com GET /wordpress/feed/ HTTP/1.0 31-3 32458 0/1/162852 W 0.04 43750 0 0.0 0.01 1758.80 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 32-3 14198 0/571/157504 W 47.36 32680 0 0.0 8.64 1733.04 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 33-3 2213 0/9/152830 W 0.32 42624 0 0.0 0.00 1782.36 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 34-3 2308 0/1/137395 W 0.06 39398 0 0.0 0.00 1433.25 64.40.121.190 zachdotsey.com GET /wordpress/ HTTP/1.0 35-3 6984 0/123/127498 W 1.88 40241 0 0.0 0.18 1350.38 206.196.125.113 zachdotsey.com GET /wordpress/?feed=rss2 HTTP/1.1 36-3 1003 0/21/116316 W 0.81 39874 0 0.0 1.15 1216.35 67.202.60.246 zachdotsey.com GET /wordpress/ HTTP/1.1 37-3 3025 0/19/106816 W 0.16 42196 0 0.0 0.28 1096.76 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 38-3 4250 1/15/101047 W 0.09 41660 0 0.5 0.01 1198.01 68.118.8.157 zachdotsey.com GET /wordpress/ HTTP/1.1 39-3 3073 0/73/91871 W 2.32 41891 0 0.0 1.81 924.03 68.118.8.157 zachdotsey.com GET /wordpress/index.php HTTP/1.1 40-3 2133 0/15/86368 W 0.20 39445 0 0.0 0.09 888.41 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 41-3 2803 0/93/79858 W 0.67 39020 0 0.0 0.69 877.38 64.40.121.190 zachdotsey.com GET /wordpress/ HTTP/1.0 42-3 3030 0/165/72532 W 35.08 41532 0 0.0 2.17 777.91 65.184.161.115 zachdotsey.com GET /wordpress/?feed=rss2 HTTP/1.1 43-3 13566 0/427/68292 W 14.81 33263 0 0.0 6.28 682.48 174.129.193.94 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 44-3 31951 0/6/65523 W 0.39 43980 0 0.0 0.15 624.37 78.31.110.126 zachdotsey.com GET /wordpress/feed HTTP/1.1 45-3 17775 0/4/64507 W 0.11 32768 0 0.0 0.00 666.08 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 46-3 17485 0/280/61533 W 30.17 16250 0 0.0 6.34 655.87 77.88.42.26 zachdotsey.com GET /robots.txt HTTP/1.1 47-3 1379 0/305/51058 W 2.50 42256 0 0.0 5.02 691.25 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 48-3 15264 0/28/49621 W 0.11 33958 0 0.0 0.02 527.41 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 49-3 20544 1/28/48765 W 1.55 40125 0 0.4 0.17 516.42 64.237.119.18 zachdotsey.com GET /wordpress/ HTTP/1.1 50-3 14318 0/408/50772 W 26.77 32722 0 0.0 7.28 509.18 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 51-3 2258 0/18/45699 W 0.08 39388 0 0.0 0.01 456.81 64.40.121.190 zachdotsey.com GET /robots.txt HTTP/1.0 52-3 3039 0/21/37675 W 0.38 42077 0 0.0 1.58 442.82 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 53-3 2747 0/3/35053 W 0.06 39268 0 0.0 0.22 371.79 64.40.121.190 zachdotsey.com GET /wordpress/ HTTP/1.0 54-3 17408 0/22/35784 W 0.26 32871 0 0.0 0.42 367.04 74.6.18.233 zachdotsey.com GET /robots.txt HTTP/1.0 55-3 4511 0/6/29200 W 0.40 41444 0 0.0 0.23 375.89 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 56-3 5534 0/54/27466 W 2.63 37695 0 0.0 0.38 305.27 68.10.212.178 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 57-3 23760 0/41/20806 W 0.23 40553 0 0.0 0.33 219.16 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 58-3 5638 0/247/17586 W 2.79 40174 0 0.0 2.66 182.34 64.237.119.18 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 59-3 31783 0/5/18112 W 0.05 40067 0 0.0 0.06 182.11 65.113.57.7 zachdotsey.com GET /wordpress/feed/ HTTP/1.0 60-3 2319 0/1/13920 W 0.03 39391 0 0.0 0.00 145.61 38.117.108.131 zachdotsey.com GET /wordpress/ HTTP/1.1 61-3 23730 0/155/12935 W 29.86 28181 0 0.0 3.25 153.35 72.30.161.222 zachdotsey.com GET /wordpress/2008/10/26/tiring-surf-sessions/ HTTP/1.0 62-3 12586 0/25/12506 W 0.14 40271 0 0.0 0.09 135.23 72.233.56.156 zachdotsey.com GET /wordpress/ HTTP/1.1 63-3 13567 0/15/10439 W 0.54 34975 0 0.0 0.10 137.24 84.217.7.221 zachdotsey.com GET /wordpress/2009/06/06/emergency-baby-call-tests/ HTTP/1.1 64-3 17776 1/161/10534 W 1.65 31758 0 0.4 2.14 128.33 208.92.218.250 zachdotsey.com GET /wordpress/ HTTP/1.1 65-3 3053 0/159/7939 W 7.25 41485 0 0.0 2.64 74.76 209.131.41.48 zachdotsey.com GET /robots.txt HTTP/1.0 66-3 5056 0/115/6934 W 1.56 37848 0 0.0 2.43 81.76 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 67-3 16535 0/160/6199 W 19.57 39641 0 0.0 2.57 67.22 64.40.121.190 zachdotsey.com GET /wordpress/ HTTP/1.0 68-3 17777 0/37/7722 W 0.24 32569 0 0.0 0.15 93.77 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 69-3 2355 0/2/5737 W 0.15 39346 0 0.0 0.03 73.54 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 70-3 10432 0/24/5080 W 0.33 36392 0 0.0 0.47 45.89 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 71-3 10433 0/12/4099 W 0.93 36365 0 0.0 0.65 41.80 68.7.141.144 zachdotsey.com GET /wordpress/2006/12/18/the-plastic-coating-on-the-end-of-as 72-3 6145 0/114/4835 W 12.61 37285 0 0.0 1.49 57.55 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 73-3 5539 0/71/4717 W 5.37 37680 0 0.0 1.63 62.39 68.10.212.178 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 74-3 17690 0/38/5004 W 0.12 32810 0 0.0 0.25 46.34 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 75-3 1528 0/38/3991 W 0.14 39764 0 0.0 0.08 41.91 64.40.121.190 zachdotsey.com GET /wordpress/ HTTP/1.0 76-3 2751 0/26/2886 W 0.22 39144 0 0.0 0.63 25.56 64.40.121.190 zachdotsey.com GET /wordpress/ HTTP/1.0 77-3 5453 0/95/6939 W 1.95 21796 0 0.0 1.09 87.49 69.147.79.37 zachdotsey.com GET /robots.txt HTTP/1.0 78-3 5061 0/192/3238 W 9.87 37110 0 0.0 2.13 39.70 78.31.110.126 zachdotsey.com GET /wordpress/feed HTTP/1.1 79-3 9955 0/143/4659 W 11.55 35985 0 0.0 7.35 64.53 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 80-3 17779 0/29/4181 W 0.33 32548 0 0.0 0.47 48.11 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 81-3 1192 0/74/2861 W 0.54 39651 0 0.0 0.42 27.09 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 82-3 13108 0/42/4116 W 0.13 35144 0 0.0 0.10 37.39 71.193.156.250 zachdotsey.com GET /wordpress/wp-login.php?loggedout=true HTTP/1.1 83-3 32180 0/0/6535 W 0.98 10753 0 0.0 0.00 105.04 142.166.3.122 zachdotsey.com GET /robots.txt HTTP/1.1 84-3 24479 0/148/4080 W 98.02 28139 0 0.0 3.46 60.62 72.30.161.222 zachdotsey.com GET /wordpress/2008/10/26/tiring-surf-sessions/ HTTP/1.0 85-3 776 1/204/8349 W 40.83 9214 0 0.4 3.57 99.11 64.237.119.18 zachdotsey.com GET /wordpress/ HTTP/1.1 86-3 13569 0/25/3218 W 10.53 34863 0 0.0 0.29 28.96 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 87-3 13110 0/2/2535 W 0.09 35187 0 0.0 0.00 26.16 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 88-3 4716 0/157/2237 W 3.75 37444 0 0.0 3.58 25.28 206.196.125.113 zachdotsey.com GET /wordpress/?feed=rss2 HTTP/1.1 89-3 19878 0/110/2928 W 10.39 31357 0 0.0 0.54 26.66 58.61.164.39 zachdotsey.com GET /robots.txt HTTP/1.1 90-3 2003 0/28/1847 W 1.81 39519 0 0.0 0.39 19.10 64.40.121.190 zachdotsey.com GET /wordpress/ HTTP/1.0 91-3 13570 0/0/1928 W 10.59 35116 0 0.0 0.00 23.59 71.193.156.250 zachdotsey.com GET /wordpress/ HTTP/1.1 92-3 16358 0/282/1910 W 33.11 32280 0 0.0 5.57 26.53 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 93-3 16367 0/50/1664 W 10.03 33243 0 0.0 0.88 17.62 174.129.193.94 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 94-3 25642 0/82/2391 W 19.24 28268 0 0.0 2.07 34.95 72.30.161.222 zachdotsey.com GET /wordpress/2008/10/26/tiring-surf-sessions/ HTTP/1.0 95-3 24480 0/261/2711 W 31.22 27528 0 0.0 3.14 43.29 75.219.249.154 zachdotsey.com GET /wordpress/tag/missy-warkentien/ HTTP/1.1 96-3 17692 0/514/1024 W 46.55 30965 0 0.0 6.50 14.37 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 97-3 10442 0/126/1041 W 1.25 35891 0 0.0 1.20 8.47 203.171.91.144 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 98-3 25643 0/103/1705 W 0.67 28008 0 0.0 2.83 20.89 74.6.18.233 zachdotsey.com GET /wordpress/2008/10/26/tiring-surf-sessions/ HTTP/1.0 99-3 22619 0/338/1657 W 27.38 28329 0 0.0 4.00 27.55 72.30.161.222 zachdotsey.com GET /robots.txt HTTP/1.0 100-3 19714 0/0/1845 W 12.66 31778 0 0.0 0.00 22.14 208.92.218.250 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 101-3 19367 1/155/1983 W 10.76 31728 0 0.4 0.15 23.66 208.92.218.250 zachdotsey.com GET /wordpress/ HTTP/1.1 102-3 26652 0/122/1187 W 41.42 26872 0 0.0 3.00 14.95 75.219.249.154 zachdotsey.com GET /wordpress/tag/missy-warkentien/ HTTP/1.1 103-3 7280 3/361/1639 W 12.44 35165 0 8.9 4.80 17.06 71.193.156.250 zachdotsey.com GET /wordpress/ HTTP/1.1 104-3 20843 0/507/949 W 99.96 28160 0 0.0 7.70 11.64 72.30.161.222 zachdotsey.com GET /wordpress/2008/10/26/tiring-surf-sessions/ HTTP/1.0 105-3 20844 0/308/1868 W 23.03 30024 0 0.0 3.00 28.11 78.31.110.126 zachdotsey.com GET /wordpress/feed HTTP/1.1 106-3 17791 1/278/1026 W 26.89 31698 0 0.4 4.18 13.29 208.92.218.250 zachdotsey.com GET /wordpress/ HTTP/1.1 107-3 17792 0/304/642 W 12.27 31388 0 0.0 2.72 8.79 58.61.164.39 zachdotsey.com GET /robots.txt HTTP/1.1 108-3 17697 0/57/515 W 0.30 32611 0 0.0 1.78 8.41 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 109-3 19715 0/611/783 W 60.52 29153 0 0.0 5.94 7.20 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 110-3 13952 0/345/614 W 5.95 33656 0 0.0 3.47 8.18 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 111-3 32009 0/75/1959 W 5.25 24778 0 0.0 1.06 22.60 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 112-3 17793 0/35/178 W 11.11 32590 0 0.0 0.44 1.92 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 113-3 19533 0/8/2004 W 0.16 16705 0 0.0 1.32 34.66 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 114-3 22622 0/304/925 W 181.63 28201 0 0.0 5.05 12.30 72.30.161.222 zachdotsey.com GET /wordpress/2008/10/26/tiring-surf-sessions/ HTTP/1.0 115-3 26656 0/38/763 W 0.55 27724 0 0.0 0.40 6.22 75.219.249.154 zachdotsey.com GET /wordpress/tag/missy-warkentien/ HTTP/1.1 116-3 31804 0/57/3039 W 2.17 10748 0 0.0 1.27 44.70 142.166.3.122 zachdotsey.com GET /wordpress/2009/07/13/millys-first-time-in-a-restaurant/ HT 117-3 26658 0/0/866 W 14.05 28069 0 0.0 0.00 11.50 74.6.18.233 zachdotsey.com GET /robots.txt HTTP/1.0 118-3 26659 0/46/646 W 4.59 27608 0 0.0 0.47 4.90 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 119-3 5457 0/86/998 W 21.35 21916 0 0.0 1.59 14.79 69.147.79.37 zachdotsey.com GET /robots.txt HTTP/1.0 120-3 9855 0/486/1470 W 58.03 18535 0 0.0 8.31 25.07 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 121-3 17802 0/8/206 W 0.05 32743 0 0.0 0.00 1.67 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 122-3 17803 0/10/141 W 0.19 32701 0 0.0 0.00 1.08 74.6.18.233 zachdotsey.com GET /wordpress/2008/12/ HTTP/1.0 123-3 19888 0/69/120 W 0.37 31389 0 0.0 0.97 1.10 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 124-3 25184 0/160/689 W 33.97 27701 0 0.0 5.24 9.16 75.219.249.154 zachdotsey.com GET /wordpress/tag/missy-warkentien/ HTTP/1.1 125-3 27208 0/615/2460 W 28.73 10736 0 0.0 7.88 44.54 142.166.3.122 zachdotsey.com GET /wordpress/2009/07/14/millys-smile/ HTTP/1.1 126-3 21945 0/281/410 W 4.56 29359 0 0.0 3.82 7.16 74.6.18.233 zachdotsey.com GET /robots.txt HTTP/1.0 127-3 25187 0/392/429 W 58.81 25928 0 0.0 7.26 7.47 75.219.249.154 zachdotsey.com GET /wordpress/ HTTP/1.1 128-3 21948 0/223/299 W 2.32 30044 0 0.0 1.29 1.77 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 129-3 27517 0/211/551 W 60.71 25918 0 0.0 4.73 10.25 75.219.249.154 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 130-3 9078 1/150/753 W 13.68 21327 0 0.4 2.78 11.88 208.92.218.250 zachdotsey.com GET /wordpress/ HTTP/1.1 131-3 20742 0/97/2004 W 1.96 15475 0 0.0 2.14 35.30 78.31.110.126 zachdotsey.com GET /wordpress/feed HTTP/1.1 132-3 12265 0/658/1664 W 62.77 16822 0 0.0 11.80 27.61 74.6.18.233 zachdotsey.com GET /robots.txt HTTP/1.0 133-3 31859 0/72/2354 W 12.54 10604 0 0.0 1.14 54.86 174.129.193.99 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 134-3 4105 0/60/658 W 12.23 22622 0 0.0 1.46 9.19 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 135-3 4106 0/195/679 W 15.18 21676 0 0.0 3.77 14.09 69.147.79.37 zachdotsey.com GET /robots.txt HTTP/1.0 136-3 25192 0/8/8 W 0.21 28835 0 0.0 0.01 0.01 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 137-3 25193 0/103/103 W 15.55 27940 0 0.0 1.20 1.20 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 138-3 25194 0/37/37 W 11.09 28465 0 0.0 1.07 1.07 38.117.108.35 zachdotsey.com GET /wordpress/feed HTTP/1.1 139-3 28509 0/64/106 W 1.12 26708 0 0.0 0.82 1.54 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 140-3 28955 0/68/68 W 2.49 26135 0 0.0 1.37 1.37 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 141-3 28957 0/59/59 W 8.77 26104 0 0.0 1.60 1.60 75.219.249.154 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 142-3 28958 0/126/126 W 4.49 26058 0 0.0 1.75 1.75 75.219.249.154 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 143-3 3101 0/26/1994 W 1.88 9152 0 0.0 0.16 32.99 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 144-3 21433 0/29/1004 W 1.06 15819 0 0.0 0.21 16.72 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 145-3 12273 0/336/786 W 29.77 18507 0 0.0 7.08 15.99 58.61.164.141 zachdotsey.com GET /wordpress/2008/11/01/early-voting/ HTTP/1.1 146-3 4109 0/65/232 W 1.64 22700 0 0.0 0.52 2.96 78.31.110.126 zachdotsey.com GET /wordpress/feed HTTP/1.1 147-3 9081 0/250/582 W 61.84 20036 0 0.0 4.35 10.27 125.244.77.2 zachdotsey.com POST /_vti_bin/_vti_aut/author.dll HTTP/1.1 148-3 21639 0/216/1433 W 8.19 14597 0 0.0 1.97 25.39 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 149-3 31137 0/1/1657 W 0.08 11318 0 0.0 0.00 29.24 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 150-3 31210 0/227/227 W 86.27 23567 0 0.0 8.50 8.50 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 151-3 4071 0/42/2081 W 12.87 8452 0 0.0 0.74 32.38 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 152-3 9904 0/31/445 W 1.53 21377 0 0.0 0.71 10.45 208.92.218.250 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 153-3 25865 0/8/1236 W 0.20 13614 0 0.0 0.13 14.50 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 154-3 9905 0/78/294 W 15.40 20614 0 0.0 1.30 4.53 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 155-3 9085 0/75/78 W 11.31 21562 0 0.0 0.88 0.88 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 156-3 18546 0/51/192 W 3.91 17077 0 0.0 0.22 4.83 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 157-3 32217 0/0/1740 W 0.24 10744 0 0.0 0.00 28.44 142.166.3.122 zachdotsey.com GET /wordpress/2009/07/11/millys-anticipation/ HTTP/1.1 158-3 14856 0/264/641 W 3.65 17963 0 0.0 2.75 9.95 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 159-3 18153 0/92/3545 W 4.82 2279 0 0.0 1.50 45.82 74.6.18.233 zachdotsey.com GET /wordpress/2008/10/24/ HTTP/1.0 160-3 31679 0/56/56 W 0.54 25195 0 0.0 0.61 0.61 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 161-3 32316 0/1/1628 W 0.05 10733 0 0.0 0.00 27.14 142.166.3.122 zachdotsey.com GET /wordpress/2009/07/12/millys-first-poop-on-daddy/ HTTP/1.1 162-3 25433 0/89/590 W 1.86 13495 0 0.0 1.50 9.58 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 163-3 19960 0/278/338 W 26.51 14965 0 0.0 3.34 3.96 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 164-3 20271 0/111/598 W 3.85 15553 0 0.0 3.49 10.91 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 165-3 13282 0/113/3121 W 3.01 4585 0 0.0 0.52 37.39 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 166-3 25641 0/227/658 W 27.64 12512 0 0.0 2.51 8.38 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 167-3 26850 0/303/816 W 6.27 12241 0 0.0 2.71 12.95 77.88.42.26 zachdotsey.com GET /robots.txt HTTP/1.1 168-3 25966 0/413/898 W 16.14 12226 0 0.0 3.66 10.86 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 169-3 25967 0/35/276 W 0.84 13409 0 0.0 1.19 3.09 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 170-3 28752 0/426/4119 W 31.27 134 0 0.0 4.26 65.53 71.127.124.128 northwoodcinemagrill.com GET / HTTP/1.1 171-3 10540 0/16/1765 W 0.22 5732 0 0.0 0.23 20.56 92.51.157.168 zachdotsey.com GET /robots.txt HTTP/1.1 172-3 4297 0/26/1208 W 0.43 8572 0 0.0 0.02 22.46 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 173-3 10221 0/140/140 W 63.93 19817 0 0.0 4.69 4.69 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 174-3 26697 0/529/845 W 24.04 11002 0 0.0 6.17 13.25 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 175-3 32336 0/224/1243 W 15.09 9411 0 0.0 2.93 20.51 207.179.181.130 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 176-3 25871 0/1/189 W 0.08 13705 0 0.0 0.01 1.38 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 177-3 25648 0/57/166 W 3.24 13460 0 0.0 0.38 3.02 58.61.164.39 zachdotsey.com GET /wordpress/2008/10/25/big-day-for-duke-sports/ HTTP/1.1 178-3 31175 0/93/975 W 26.19 10624 0 0.0 3.61 13.03 174.129.193.99 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 179-3 10228 1/34/34 W 0.26 21297 0 0.4 1.60 1.60 208.92.218.250 zachdotsey.com GET /wordpress/ HTTP/1.1 180-3 10229 0/235/235 W 11.14 19431 0 0.0 6.96 6.96 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 181-3 14002 0/180/2169 W 10.99 4030 0 0.0 1.11 18.94 65.184.161.115 zachdotsey.com GET /wordpress/ HTTP/1.1 182-3 31180 0/307/460 W 38.99 9753 0 0.0 6.36 7.88 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 183-3 10232 1/28/28 W 1.84 21357 0 0.4 0.26 0.26 208.92.218.250 zachdotsey.com GET /wordpress/ HTTP/1.1 184-3 31929 0/46/714 W 0.38 10749 0 0.0 0.27 10.75 142.166.3.122 zachdotsey.com GET /wordpress/2009/07/15/andre-dawkins-to-duke-a-year-early/ H 185-3 8930 0/201/1413 W 11.72 6142 0 0.0 0.90 18.43 92.51.157.168 zachdotsey.com GET /robots.txt HTTP/1.1 186-3 31931 0/314/471 W 47.33 9256 0 0.0 9.23 13.30 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 187-3 1832 0/78/850 W 21.91 9234 0 0.0 1.14 8.29 64.237.119.18 zachdotsey.com GET /wordpress/feed/ HTTP/1.1 188-3 31181 0/121/556 W 5.97 10756 0 0.0 2.10 13.38 142.166.3.122 zachdotsey.com GET /robots.txt HTTP/1.1 189-3 9444 0/251/1730 W 5.98 5671 0 0.0 2.12 28.18 65.184.161.115 zachdotsey.com POST /wordpress/wp-admin/admin-ajax.php HTTP/1.1 190-3 31932 0/25/169 W 2.85 10769 0 0.0 0.73 2.87 142.166.3.122 zachdotsey.com GET /robots.txt HTTP/1.1 191-3 25658 0/6/6 W 0.10 13734 0 0.0 0.04 0.04 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 192-3 9884 0/136/1248 W 1.37 5630 0 0.0 1.32 14.36 65.184.161.115 zachdotsey.com GET /wordpress/ HTTP/1.1 193-3 324 0/37/116 W 5.95 10106 0 0.0 1.27 2.86 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 194-3 19087 0/756/2584 W 67.99 1279 0 0.0 9.26 35.86 78.31.110.126 zachdotsey.com GET /wordpress/feed HTTP/1.1 195-3 20058 0/722/2684 W 62.21 1123 0 0.0 7.96 30.71 206.196.125.113 zachdotsey.com GET /wordpress/?feed=rss2 HTTP/1.1 196-3 2425 1/38/56 W 2.18 9154 0 0.4 0.47 0.59 64.237.119.18 zachdotsey.com GET /wordpress/ HTTP/1.1 197-3 28767 0/341/3091 W 12.60 64 0 0.0 2.89 33.12 65.184.161.115 myvenue.venuecom.com GET / HTTP/1.1 198-3 4816 0/260/942 W 26.04 7324 0 0.0 2.25 10.97 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 199-3 21139 0/567/2438 W 121.57 18 0 0.0 10.32 43.01 208.59.151.137 webdav.venuecom.com GET /calendar/dnewcorn/calendars/Web%20Demos.ics HTTP/1.1 200-3 4311 0/55/188 W 4.84 8571 0 0.0 1.31 3.77 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 201-3 5891 0/15/205 W 0.11 7957 0 0.0 1.61 6.07 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 202-3 30984 0/0/2581 W 42.72 40 0 0.0 0.00 31.17 207.210.148.62 marketplace-mail.com GET /checklist.php HTTP/1.1 203-3 2435 1/40/42 W 19.37 9184 0 0.4 2.43 2.44 64.237.119.18 zachdotsey.com GET /wordpress/ HTTP/1.1 204-3 29989 0/26/2777 W 30.30 78 0 0.0 1.51 38.95 198.178.147.1 raleighwoodmovies.com GET / HTTP/1.1 205-3 19694 0/0/1370 W 0.14 2633 0 0.0 0.00 19.70 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 206-3 19406 0/202/1756 W 10.98 2238 0 0.0 1.83 20.94 74.6.18.233 zachdotsey.com GET /wordpress/2008/10/24/ HTTP/1.0 207-3 19695 0/937/2326 W 42.78 482 0 0.0 8.58 23.44 76.182.6.152 raleighwoodmovies.com GET / HTTP/1.1 208-3 11763 0/129/802 W 14.71 4766 0 0.0 2.16 9.27 65.184.161.115 zachdotsey.com GET /wordpress/?feed=rss2 HTTP/1.1 209-3 2442 0/26/27 W 9.27 9236 0 0.0 0.54 0.54 77.88.42.26 zachdotsey.com GET /robots.txt HTTP/1.1 210-3 2447 0/94/95 W 7.65 9136 0 0.0 0.73 0.73 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 211-3 3914 0/80/100 W 6.41 8330 0 0.0 0.48 0.61 78.31.110.126 zachdotsey.com GET /wordpress/feed HTTP/1.1 212-3 11674 0/0/519 W 14.19 5522 0 0.0 0.00 7.40 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 213-3 7627 0/70/211 W 2.47 6989 0 0.0 1.77 3.11 77.88.42.26 zachdotsey.com GET /robots.txt HTTP/1.1 214-3 29724 0/557/557 W 80.97 9376 0 0.0 7.93 7.93 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 215-3 19410 0/980/2490 W 162.44 138 0 0.0 7.87 26.41 198.178.147.1 raleighwoodmovies.com GET /printable.php HTTP/1.1 216-3 9451 0/9/405 W 0.16 6415 0 0.0 0.06 2.35 66.249.66.2 zachdotsey.com GET /robots.txt HTTP/1.1 217-3 11764 0/9/695 W 0.13 5407 0 0.0 0.08 9.91 98.122.166.13 zachdotsey.com GET /wordpress/ HTTP/1.1 218-3 20889 0/46/1701 W 0.18 2217 0 0.0 0.37 26.16 74.6.18.233 zachdotsey.com GET /wordpress/2008/10/24/ HTTP/1.0 219-3 17642 0/511/1352 W 8.05 2259 0 0.0 2.89 13.17 74.6.18.233 zachdotsey.com GET /wordpress/2008/10/24/ HTTP/1.0 220-3 19411 0/757/1910 W 29.09 1126 0 0.0 5.47 18.72 65.184.161.115 zachdotsey.com GET /wordpress/ HTTP/1.1 221-3 19412 0/284/1153 W 5.34 2085 0 0.0 3.74 22.78 74.6.18.233 zachdotsey.com GET /wordpress/2008/10/24/ HTTP/1.0 222-3 19696 0/885/1794 W 54.08 237 0 0.0 15.96 25.04 69.86.60.67 webdav.venuecom.com GET /calendar/lswuser/private/calendars/Prospecting.ics HTTP/1. 223-3 19697 0/57/1272 W 0.58 2409 0 0.0 0.38 11.08 74.6.18.233 zachdotsey.com GET /robots.txt HTTP/1.0 224-3 20500 0/226/1395 W 0.82 2147 0 0.0 2.12 11.41 74.6.18.233 zachdotsey.com GET /wordpress/2008/10/24/ HTTP/1.0 225-3 21060 0/776/1869 W 36.25 475 0 0.0 10.92 24.86 78.31.110.127 allnews.allchoiceinsurance.com GET /feed/ HTTP/1.1 226-3 15942 8/360/708 W 10.60 2428 0 29.1 3.08 6.61 65.184.161.115 zachdotsey.com GET /wordpress/ HTTP/1.1 227-3 18059 1/9/402 W 2.48 3120 0 419.1 0.83 4.29 24.199.245.190 zachdotsey.com GET /wordpress/ HTTP/1.1 228-3 3929 0/102/102 W 16.70 8332 0 0.0 1.24 1.24 76.13.20.61 zachdotsey.com GET /robots.txt HTTP/1.0 229-3 7814 0/343/396 W 2.93 6380 0 0.0 1.71 1.86 98.122.166.13 zachdotsey.com GET /wordpress/ HTTP/1.1 230-3 20958 0/746/2021 W 86.23 121 0 0.0 11.92 25.76 72.30.87.105 baucomsbest.com GET /index.php HTTP/1.0 231-3 20897 1/765/1673 K 43.84 4 15 10.6 7.17 17.99 24.163.58.115 myvenue.venuecom.com GET /editor/images/progress-bar-back.gif HTTP/1.1 ... 

在你说出你有什么模块之后,我注意到你说过EAccelerator,它有一个错误(Bug跟踪链接在这里 ),导致Apache在永无止境的循环中locking,并使服务器几乎停顿,尝试禁用它,看看会发生什么。

在问题时间你看到很多 POST /wordpress/wp-admin/admin-ajax.php请求吗? 如果是这样的话,可能是由于垃圾邮件发送者试图蛮横的行为,通过故意的拒绝服务尝试,或者只是因为这个站点在某个地方忙碌一次又一次地发生(你的日志显示了很多来自同一个地方的请求 – 网站引用者?)。

它也可能是一个数据库locking的问题,即一个请求已经要求在一个频繁访问的表上使用全部的deny-read-until-done锁来执行一个长操作,而其他的请求只是等待锁被释放which may happen naturally, or as a result of you killing Apache and therefore the connection that is holding the lock(s). If the site is busy enough then this can create significant load as lots of Apache processes get created which then sit consuming memory until they can continue past the lock – if your Apache config is such that a relatively unconstrained number of child processes can be created then the system could easily start thrashing swap at this point.

To see if the load is being caused by CPU use, IO due to DB access and such, or IO due to swap thrashing, you'll need to investigate the output of htop (or just plain top), free, and the other usual system load monitoring utils. This may give more clues as to where the root of the issue might be found.

That definitely looks like a slowloris DoS attack against one of the sites you host.

Find more details, as well as Debian and Red-Hat solutions here

One of the other symptoms of that type of attack will be the appearance of dozens/hundreds of 400 errors in your Apache logs (if/after the attack ends – read more at ha.ckers.org/slowloris/)

You could try setting the TimeOut value in apache to something quite draconian like 20 seconds. This should finish the connection if certain actions take longer to complete. This is sort of papering over the problem, rather than solving it. It should probably be used to debug if the problem is due to something taking too long, rather than as a final solution.

Are your PHP scripts making any database requests? If so, verify that the database is responding in a timely manner.

David covers most of the essentials in his response .

Also, similar to the DB locking issue, could be a call on the backend to an external resource that's not responding or responding slowly. For example, if you had a CGI that did a LWP to example.net, Apache would wait until timeout or response. This can cause a resource exhaustion as well.

How often do you ssh into the box? I had a problem, as weird as it sounds, that ssh would actually kill the apache service. I would ssh in and everything would just lock up. I haven't vpn'ed into the box in months and there has been no problem at all. It didn't happen everytime. But I know there was a correlation.

When dealing with any performance problem, you have to know the input. Configure your switch to mirror the webserver port to a sniffer server where you record all HTTP traffic. Try to correlate the traffic with the problem and try to reproduce the problem on a test machine.

Next step is to see what is happening with those processes. Use strace and see in which system call they are waiting. Use lsof to find out the file descriptors involved in the strace. ls -l /proc/<PID>/fd is useful too.

As chronos said, it could be a slowloris atack.