由于“太多打开的文件”,Apache每晚崩溃

我正在Joyent上运行一台SmartMachine。 我相信这些是运行Solaris的某种虚拟机。 我们在运行Apache,PHP和MySQL的机器上有一个Web应用程序。 它处理我们适度的stream量很好。 然而,每天晚上,我们已经走了。 该网站将开始返回403 Forbidden错误,直到Apache重新启动。 我快速看一下Apache的错误日志显示如下:

[Tue Oct 26 23:13:00 2010] [error] server reached MaxClients setting, consider raising the MaxClients setting [Wed Oct 27 13:09:40 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for reading (fetch) [Wed Oct 27 13:09:40 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for writing (store) [Wed Oct 27 13:09:40 2010] [error] [client 98.25.133.36] PHP Fatal error: Unknown: Failed opening required '/home/jill/web/content/index.php' (include_path='.:/opt/local/lib/php') in Unknown on line 0, referer: https://[redacted]/presentations/present# [Wed Oct 27 13:09:42 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for reading (fetch) [Wed Oct 27 13:09:42 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for writing (store) [Wed Oct 27 13:09:42 2010] [crit] [client 68.193.4.75] (24)Too many open files: /home/jill/web/content/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: https://[redacted]/presentations/present# [Wed Oct 27 13:09:43 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for reading (fetch) [Wed Oct 27 13:09:43 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for writing (store) [Wed Oct 27 13:09:43 2010] [crit] [client 72.28.224.201] (24)Too many open files: /home/jill/web/content/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: https://[redacted]/presentations/present# [Wed Oct 27 13:09:44 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for reading (fetch) [Wed Oct 27 13:09:44 2010] [error] (24)Too many open files: Cannot open SSLSessionCache DBM file `/var/run/ssl_scache' for writing (store) [Wed Oct 27 13:09:44 2010] [crit] [client 72.28.224.201] (24)Too many open files: /home/jill/web/content/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: https://[redacted]/presentations/present# 

最后三行然后重复每个请求到服务器。 我真的不知如何防止这种情况发生。 我尝试增加可以使用prctl打开的文件的数量,但我不能正确使用它,因为当我尝试将其设置为65.5K时,prctl基本返回1.02K。 我甚至不确定这是否是一个合理的解决scheme:

 prctl -i process -n process.max-file-descriptor `pgrep httpd` process: 18284: /opt/local/sbin/httpd -k start NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT process.max-file-descriptor basic 1.02K - deny 18284 privileged 65.5K - deny - system 2.15G max deny - process: 18285: /opt/local/sbin/httpd -k start NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT process.max-file-descriptor basic 1.02K - deny 18285 privileged 65.5K - deny - system 2.15G max deny - 

那么追踪和解决这个问题的最好方法是什么?

UPDATE

这里是根httpd进程的pfiles的输出。

 [root@fe5txrad ~]# pfiles 18269 18269: /opt/local/sbin/httpd -k start Current rlimit: 1024 file descriptors 0: S_IFCHR mode:0666 dev:304,8 ino:3020727013 uid:0 gid:3 rdev:13,2 O_RDONLY /dev/null 1: S_IFCHR mode:0666 dev:304,8 ino:3020727013 uid:0 gid:3 rdev:13,2 O_WRONLY|O_CREAT|O_TRUNC /dev/null 2: S_IFREG mode:0640 dev:182,65550 ino:362926 uid:0 gid:0 size:20551848 O_WRONLY|O_APPEND|O_CREAT|O_LARGEFILE /var/log/httpd/error.log 3: S_IFDOOR mode:0444 dev:313,0 ino:38 uid:0 gid:0 size:0 O_RDONLY|O_LARGEFILE FD_CLOEXEC door to nscd[18176] /var/run/name_service_door 4: S_IFSOCK mode:0666 dev:311,0 ino:43693 uid:0 gid:0 size:0 O_RDWR|O_NONBLOCK FD_CLOEXEC SOCK_STREAM SO_REUSEADDR,SO_KEEPALIVE,SO_SNDBUF(49152),SO_RCVBUF(49152) sockname: AF_INET 0.0.0.0 port: 80 5: S_IFSOCK mode:0666 dev:311,0 ino:42512 uid:0 gid:0 size:0 O_RDWR|O_NONBLOCK FD_CLOEXEC SOCK_STREAM SO_REUSEADDR,SO_KEEPALIVE,SO_SNDBUF(49152),SO_RCVBUF(49152) sockname: AF_INET 0.0.0.0 port: 443 6: S_IFIFO mode:0000 dev:301,0 ino:8763127 uid:0 gid:0 size:0 O_RDWR|O_NONBLOCK FD_CLOEXEC 7: S_IFIFO mode:0000 dev:301,0 ino:8763127 uid:0 gid:0 size:0 O_RDWR FD_CLOEXEC 8: S_IFREG mode:0640 dev:182,65550 ino:362927 uid:0 gid:0 size:1450493 O_WRONLY|O_APPEND|O_CREAT|O_LARGEFILE FD_CLOEXEC /var/log/httpd/access.log 9: S_IFREG mode:0644 dev:182,65550 ino:369102 uid:1000 gid:1000 size:528239971 O_WRONLY|O_APPEND|O_CREAT|O_LARGEFILE FD_CLOEXEC /home/jill/logs/access_log 10: S_IFREG mode:0644 dev:182,65550 ino:369102 uid:1000 gid:1000 size:528239971 O_WRONLY|O_APPEND|O_CREAT|O_LARGEFILE FD_CLOEXEC /home/jill/logs/access_log 11: S_IFREG mode:0644 dev:308,39 ino:3386326219 uid:0 gid:0 size:0 O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE FD_CLOEXEC 12: S_IFREG mode:0644 dev:308,39 ino:3088492558 uid:0 gid:0 size:0 O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE FD_CLOEXEC advisory write lock set by process 7350 13: S_IFSOCK mode:0666 dev:311,0 ino:6452 uid:0 gid:0 size:0 O_RDWR FD_CLOEXEC SOCK_STREAM SO_SNDBUF(16384),SO_RCVBUF(5120) sockname: AF_UNIX 

当你开始得到这些错误时,你没有提到你有多less连接。

您可以使用如下工具/命令:

 ps -ef | grep apache | wc -l lsof -p <apache pid> netstat -anp | grep 80 | grep -i ESTABLISHED | wc -l 

第一个命令告诉你系统上有多less个apache进程。 第二个命令告诉你有多less打开的文件/连接apache进程。 当然,你需要用真正的价值取代。 第三个命令告诉你有多lessbuild立的连接。

我曾经被类似的东西烧毁。 还有一件事你可能要检查 – 即PHP不吸取文件。 如果PHP正在将会话信息存储在磁盘上,请求的突然超载可能会吸收所有文件。

如果你有良好的内存和CPU,你可以增加没有。 通过ulimit命令打开文件限制

ulimit -n 32667(也可能比这更高)

我认为错误“太多打开的文件”可能是一个误用。 有太多的参考资料无法读取/写入到/ var / run这是solaris的虚拟内存。

几个问题:

你有没有正确configurationApache来处理负载的可用资源量? 你允许多less个子线程fork? 每个线程占用多less内存? 你configuration了多less交换?

一个方便的一个class轮,找出有多less内存httpd进程总共使用:

 for i in `pgrep httpd`; do pmap -x $i | nawk '/total/ {a+=$4} END { print a /1024" MB" }'; done | nawk '{a+=$1} END { print a " MB" }' 

我敢肯定,有人可以把这更雄辩,但它适用于我。 心连心。

如果你有良好的内存和CPU,你可以增加没有。 通过ulimit命令打开文件限制

ulimit -n 32667(也可能比这更高)

我知道这是一个古老的问题,但另一个问题是对这个问题有一个非常恰当的回应,我想知道它是否也适用于你:

https://superuser.com/a/829413/196842

基本上,当debugging客户端没有打开时,Xdebug正在泄漏连接监听器的文件描述符(对不起,如果这在技术上是不正确的,这就是主意)。 要解决此问题,请确保在远程debugging器启动debugging会话时debugging器客户端处于打开状态。 当然,更好的解决scheme是修复开发人员的错误。