Apache + mod_jk + GlassFish v3.1集群的大型并发用户性能问题

我在GlassFish v3.1上运行一个java ee 6 ear应用程序(每个包含两个实例的2个集群),由Apache v2.2使用mod_jk进行负载均衡 – 所有服务器位于同一台服务器上(Windows Server 2003 R2,Intel Xeon CPU x5670 @ 2.93Ghz,6GB RAM,2 cpus)。

networking应用程序被大约100个用户访问。 当他们都试图在每天早上8点的同一时间访问它时,在尝试访问主jsf主页时,响应非常缓慢。

除此之外,我经常看到httpd进程的CPU使用率高达99%,我开始在mod_jk.log文件中看到错误。

[Wed Jun 08 08:25:43 2011] [9380:8216] [info] ajp_process_callback::jk_ajp_common.c (1885): Writing to client aborted or client network problems [Wed Jun 08 08:25:43 2011] [9380:8216] [info] ajp_service::jk_ajp_common.c (2543): (myAppLocalInstance4) sending request to tomcat failed (unrecoverable), because of client write error (attempt=1) 

关于如何改善这个问题的build议?

Apacheconfiguration大部分是默认的,如下所示

 ServerRoot "C:/Program Files/Apache Software Foundation/Apache2.2" Listen 80 LoadModule actions_module modules/mod_actions.so LoadModule alias_module modules/mod_alias.so LoadModule asis_module modules/mod_asis.so LoadModule auth_basic_module modules/mod_auth_basic.so LoadModule authn_default_module modules/mod_authn_default.so LoadModule authn_file_module modules/mod_authn_file.so LoadModule authz_default_module modules/mod_authz_default.so LoadModule authz_groupfile_module modules/mod_authz_groupfile.so LoadModule authz_host_module modules/mod_authz_host.so LoadModule authz_user_module modules/mod_authz_user.so LoadModule autoindex_module modules/mod_autoindex.so LoadModule cgi_module modules/mod_cgi.so LoadModule dir_module modules/mod_dir.so LoadModule env_module modules/mod_env.so LoadModule include_module modules/mod_include.so LoadModule isapi_module modules/mod_isapi.so LoadModule log_config_module modules/mod_log_config.so LoadModule mime_module modules/mod_mime.so LoadModule negotiation_module modules/mod_negotiation.so LoadModule setenvif_module modules/mod_setenvif.so <IfModule !mpm_netware_module> <IfModule !mpm_winnt_module> User daemon Group daemon </IfModule> </IfModule> DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs" <Directory /> Options FollowSymLinks AllowOverride None Order deny,allow Deny from all </Directory> <Directory "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs"> Options Indexes FollowSymLinks AllowOverride None Order allow,deny Allow from all </Directory> <IfModule dir_module> DirectoryIndex index.html </IfModule> <FilesMatch "^\.ht"> Order allow,deny Deny from all Satisfy All </FilesMatch> ErrorLog "logs/error.log" LogLevel warn <IfModule log_config_module> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common <IfModule logio_module> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio </IfModule> CustomLog "logs/access.log" common </IfModule> <IfModule alias_module> ScriptAlias /cgi-bin/ "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin/" </IfModule> <Directory "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin"> AllowOverride None Options None Order allow,deny Allow from all </Directory> DefaultType text/plain <IfModule mime_module> TypesConfig conf/mime.types AddType application/x-compress .Z AddType application/x-gzip .gz .tgz </IfModule> Include conf/extra/httpd-mpm.conf <IfModule ssl_module> SSLRandomSeed startup builtin SSLRandomSeed connect builtin </IfModule> LoadModule jk_module modules/mod_jk.so JkWorkersFile conf/workers.properties JkLogFile logs/mod_jk.log JkLogLevel info JkLogStampFormat "[%a %b %d %H:%M:%S %Y] " JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories JkRequestLogFormat "%w %V %T" JkMount /myApp/* loadbalancerLocal JkMount /myAppRemote/* loadbalancerRemote JkMount /myApp loadbalancerLocal JkMount /myAppRemote loadbalancerRemote 

workers.propertiesconfiguration文件是:

 worker.list=loadbalancerLocal,loadbalancerRemote worker.myAppLocalInstance1.type=ajp13 worker.myAppLocalInstance1.host=localhost worker.myAppLocalInstance1.port=8109 worker.myAppLocalInstance1.lbfactor=1 worker.myAppLocalInstance1.socket_keepalive=1 worker.myAppLocalInstance1.socket_timeout=1000 worker.myAppLocalInstance2.type=ajp13 worker.myAppLocalInstance2.host=localhost worker.myAppLocalInstance2.port=8209 worker.myAppLocalInstance2.lbfactor=1 worker.myAppLocalInstance2.socket_keepalive=1 worker.myAppLocalInstance2.socket_timeout=1000 worker.myAppLocalInstance3.type=ajp13 worker.myAppLocalInstance3.host=localhost worker.myAppLocalInstance3.port=8309 worker.myAppLocalInstance3.lbfactor=1 worker.myAppLocalInstance3.socket_keepalive=1 worker.myAppLocalInstance3.socket_timeout=1000 worker.myAppLocalInstance4.type=ajp13 worker.myAppLocalInstance4.host=localhost worker.myAppLocalInstance4.port=8409 worker.myAppLocalInstance4.lbfactor=1 worker.myAppLocalInstance4.socket_keepalive=1 worker.myAppLocalInstance4.socket_timeout=1000 worker.myAppRemoteInstance1.type=ajp13 worker.myAppRemoteInstance1.host=localhost worker.myAppRemoteInstance1.port=8509 worker.myAppRemoteInstance1.lbfactor=1 worker.myAppRemoteInstance1.socket_keepalive=1 worker.myAppRemoteInstance1.socket_timeout=1000 worker.myAppRemoteInstance2.type=ajp13 worker.myAppRemoteInstance2.host=localhost worker.myAppRemoteInstance2.port=8609 worker.myAppRemoteInstance2.lbfactor=1 worker.myAppRemoteInstance2.socket_keepalive=1 worker.myAppRemoteInstance2.socket_timeout=1000 worker.myAppRemoteInstance3.type=ajp13 worker.myAppRemoteInstance3.host=localhost worker.myAppRemoteInstance3.port=8709 worker.myAppRemoteInstance3.lbfactor=1 worker.myAppRemoteInstance3.socket_keepalive=1 worker.myAppRemoteInstance3.socket_timeout=1000 worker.myAppRemoteInstance4.type=ajp13 worker.myAppRemoteInstance4.host=localhost worker.myAppRemoteInstance4.port=8809 worker.myAppRemoteInstance4.lbfactor=1 worker.myAppRemoteInstance4.socket_keepalive=1 worker.myAppRemoteInstance4.socket_timeout=1000 worker.loadbalancerLocal.type=lb worker.loadbalancerLocal.sticky_session=True worker.loadbalancerLocal.balance_workers=myAppLocalInstance1,myAppLocalInstance2,myAppLocalInstance3,myAppLocalInstance4 worker.loadbalancerRemote.type=lb worker.loadbalancerRemote.balance_workers=myAppRemoteInstance1,myAppRemoteInstance2,myAppRemoteInstance3,myAppRemoteInstance4 worker.loadbalancerRemote.sticky_session=True 

我想知道为什么你在同一台机器上运行这么多的实例,但我想你已经testing了这个设置,发现它提供了最高的性能。 另外,由于没有提到EAR(数据库?)的后端,我们假设这里没有问题。

因此,我会接近这个与glassfish / mod_jk设置无关的东西。 检查100个用户访问同一页面时实际发生的情况。 客户端打开了多less个并行连接,这与Apache的MaxClient有什么关系? 静态资源是否被客户端caching? 你正在发送Etag / Last-Modified / Cache-Control头文件吗? 你可以减less请求的数量(用YSlow检查页面)?

接下来,获取Apache提供的静态资源,而不是Glassfish应用服务器。 这将减轻appservers和loadbalancer,并释放实际dynamic页面创build的插槽。 为此,从EAR中提取CSS / JS文件和图像,并将它们放置到目录中的apache的文档根目录中(例如/static/ )。 然后确保客户端以这种方式请求资源,或者使用RewriteEngine来相应地映射请求。

如果页面创build很昂贵,但结果是可以caching的,可以考虑在Apache代理之前放置一个Varnish实例。 你需要首先控制caching头。

祝你好运!