我们经历了一些非常高的服务器负载:
1:49 pm 2天1:51,1位用户,平均负载:79.05,101.35,111.53
那么该网站崩溃。 我的意思是,networking不加载,我无法访问通过SSH或FTP,我们必须做一个硬重置。
它随机发生。
崩溃时间错误日志(类似这样的,但重复了数百次):
[Sat Jul 09 13:02:54 2011] [notice] child pid 1966 exit signal Segmentation fault (11) [Sat Jul 09 13:02:57 2011] [notice] child pid 1967 exit signal Segmentation fault (11) [Sat Jul 09 13:02:57 2011] [notice] child pid 1969 exit signal Segmentation fault (11) [Sat Jul 09 13:02:57 2011] [notice] child pid 1970 exit signal Segmentation fault (11) [Sat Jul 09 13:02:57 2011] [notice] child pid 1971 exit signal Segmentation fault (11) [Sat Jul 09 13:02:57 2011] [notice] child pid 1972 exit signal Segmentation fault (11) [Sat Jul 09 13:02:57 2011] [notice] child pid 1973 exit signal Segmentation fault (11) [Sat Jul 09 13:02:57 2011] [notice] child pid 1974 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1976 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1977 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1978 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1979 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1980 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1981 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1982 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1983 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1984 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1985 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1986 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1987 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1988 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1989 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1990 exit signal Segmentation fault (11) [Sat Jul 09 13:02:58 2011] [notice] child pid 1991 exit signal Segmentation fault (11) [Sat Jul 09 13:03:16 2011] [notice] child pid 1992 exit signal Segmentation fault (11) [Sat Jul 09 13:03:17 2011] [notice] child pid 1993 exit signal Segmentation fault (11) [Sat Jul 09 13:03:21 2011] [notice] child pid 1994 exit signal Segmentation fault (11) [Sat Jul 09 13:03:21 2011] [notice] child pid 1995 exit signal Segmentation fault (11)
碰撞小时访问日志:
::1 - - [09/Jul/2011:12:54:07 +0200] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.10 (Linux/SUSE) (internal dummy connection)" ::1 - - [09/Jul/2011:13:38:48 +0200] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.2.10 (Linux/SUSE) (internal dummy connection)"
我们的用户数量不是很多,所以这是一个小小的挑战。 我在哪里可以find关于崩溃的一些信息?,我们有一个灯架构。
提前致谢
configurationApache使其在segfault上生成一个核心转储。 之后,尝试用gdb进行debugging,看看是什么原因。
这是否正常发生? 或者是随机的?
您是否检查Web服务器上的访问和错误日志?
你看看iostat,看是否负载是由磁盘访问或networking访问或…?
顶部是否告诉你一个特定的过程是否在吃资源?
系统日志会告诉你什么?
当你说崩溃时,你是什么意思? 内核恐慌? 应用程序有一个错误? 服务器只是停止响应? 控制台工作,但networking不?
我们需要更多的信息,甚至冒险猜测。 当发生这种情况的时候,你需要缩小范围,如果发生的时候有一个模式,以及它发生的是什么子系统。