我们使用Lua（openresty bundle）设置Nginx作为本地caching节点到我们的文件共享服务器，我们将文件分隔为“每个50MB”（通过这种方法），并将它们存储在caching中以提高效率。 在低stream量下，它工作正常，但随着caching文件和负载的增加（即使不是很高），由于大部分时间系统购买超过80％，caching将无响应。 所以在这种情况下可能会有什么样的performance杀手
虽然我们尝试调整几个参数（即caching目录级别，RAID参数），但我没有给出最佳解决scheme
PS。 caching中只有10000个文件后服务器上出现约300个连接/秒的症状
caching服务器规格
1xCPU 2.5 Ghz 12 Cores 128GB RAM 10x500GB Samsung SSD RAID0 (128KB chuck s) storage linux Os -CentOS 6.6 64bit File system ext4 4k block Nginx conf
worker_processes auto; events { use epoll; worker_connections 1024; multi_accept on; } http { include /usr/local/openresty/nginx/conf/mime.types; proxy_cache_path /mnt/cache/ levels=2:2:2 keys_zone=default:1000m loader_threshold=100 loader_files=2000 loader_sleep=10 inactive=1y max_size=3500000m; proxy_temp_path /mnt/temp2 2 2; client_body_temp_path /mnt/temp 2 2; limit_conn_zone $remote_addr$uri zone=addr:100m; map $request_method $disable_cache { HEAD 1; default 0; } lua_package_path "/opt/ranger/external/lua-resty-http/lib/?.lua;/opt/ranger/external/nginx_log_by_lua/?.lua;/opt/ranger/external/bitset/lib/?.lua;;"; lua_shared_dict file_dict 50M; lua_shared_dict log_dict 100M; lua_shared_dict cache_dict 100M; lua_shared_dict chunk_dict 100M; proxy_read_timeout 20s; proxy_send_timeout 25s; reset_timedout_connection on; init_by_lua_file '/opt/ranger/init.lua'; # Server that has the lua code and will be accessed by clients server { listen 80 default; server_name _; server_name_in_redirect off; set $ranger_cache_status $upstream_cache_status; lua_check_client_abort on; lua_code_cache on; resolver ----; server_tokens off; resolver_timeout 1s; location / { try_files $uri $uri/ index.html; } location ~* ^/download/ { lua_http10_buffering off; content_by_lua_file '/opt/ranger/content.lua'; log_by_lua_file '/opt/ranger/log.lua'; limit_conn addr 2; } } # Server that works as a backend to the lua code server { listen 8080; server_tokens off; resolver_timeout 1s; location ~* ^/download/(.*?)/(.*?)/(.*) { set $download_uri $3; set $download_host $2; set $download_url http://$download_host/$download_uri?$args; proxy_no_cache $disable_cache; proxy_cache_valid 200 1y; proxy_cache_valid 206 1y; proxy_cache_key "$scheme$proxy_host$uri$http_range"; proxy_cache_use_stale error timeout http_502; proxy_cache default; proxy_cache_min_uses 1; proxy_pass $download_url; } } }
感谢@myaut的指导，我_spin_lock_irqsave了一下_spin_lock_irqsave ，原来和内核本身有关，而不是Nginx。
根据这篇文章，问题可以通过禁用修复问题的RedHat透明巨大页面function来解决。
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled