ps aux挂在高CPU / IO与Java进程

我有一些与java进程和nrpe检查问题。 我们有一些进程有时在32核心系统上使用1000％的CPU。 该系统是非常敏感的，直到你做一个

ps aux 

或者尝试在/ proc / pid中做任何事情

 [[email protected] /proc/18679]# ls hangs.. 

一个ps aux

 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0 stat("/dev/pts1", 0x7fffb8526f00) = -1 ENOENT (No such file or directory) stat("/dev/pts", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0 readlink("/proc/15693/fd/2", "/dev/pts/1", 127) = 10 stat("/dev/pts/1", {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0 write(1, "root 15693 15692 0 06:25 pt"..., 55root 15693 15692 0 06:25 pts/1 00:00:00 ps -Af ) = 55 stat("/proc/18679", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0 open("/proc/18679/stat", O_RDONLY) = 5 read(5, "18679 (java) S 1 18662 3738 3481"..., 1023) = 264 close(5) = 0 open("/proc/18679/status", O_RDONLY) = 5 read(5, "Name:\tjava\nState:\tS (sleeping)\nT"..., 1023) = 889 close(5) = 0 open("/proc/18679/cmdline", O_RDONLY) = 5 read(5, 

Java进程正在工作，并将完成很好，但问题是这使得我们的监控疯狂思考过程中，因为它超时等待ps辅助完成。

• 使用nginx作为主要Web服务器的缺点？
• 请求频率下降时为什么响应时间会爆炸？
• 如何减lessTIME_WAIT中的套接字数量？

我试过做类似的事情

  nice -19 ionice -c1 /usr/lib64/nagios/plugins/check_procs -w 1:1 -c 1:1 -a 'diamond' -u root -t 30 

没有运气

编辑

系统规格

• 32核心Intel（R）Xeon（R）CPU E5-2650 0 @ 2.00GHz
• 128公斤的RAM
• 12个4Tb 7200驱动器
• CentOS 6.5
• 我不确定模型，但供应商是SuperMicro

发生这种情况时的负载是1分钟90-160左右。

奇怪的是我可以进入任何其他/ proc / PID＃它工作得很好。 系统是响应当我ssh英寸当我们得到警报的高负荷我可以ssh正确的罚款。

另一个编辑

我一直在使用调度程序的截止date

 [[email protected] ~]# for i in {a..m}; do cat /sys/block/sd${i}/queue/scheduler; done noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq 

山看起来像

 [[email protected] ~]# mount /dev/sda3 on / type ext4 (rw,noatime,barrier=0) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext2 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/sdb1 on /disk1 type xfs (rw,nobarrier) /dev/sdc1 on /disk2 type xfs (rw,nobarrier) /dev/sdd1 on /disk3 type xfs (rw,nobarrier) /dev/sde1 on /disk4 type xfs (rw,nobarrier) /dev/sdf1 on /disk5 type xfs (rw,nobarrier) /dev/sdg1 on /disk6 type xfs (rw,nobarrier) /dev/sdh1 on /disk7 type xfs (rw,nobarrier) /dev/sdi1 on /disk8 type xfs (rw,nobarrier) /dev/sdj1 on /disk9 type xfs (rw,nobarrier) /dev/sdk1 on /disk10 type xfs (rw,nobarrier) /dev/sdl1 on /disk11 type xfs (rw,nobarrier) /dev/sdm1 on /disk12 type xfs (rw,nobarrier) 

好吧，我试图安装调整，并设置为吞吐量性能。

 [[email protected] ~]# tuned-adm profile throughput-performance Switching to profile 'throughput-performance' Applying deadline elevator: sda sdb sdc sdd sde sdf sdg sdh[ OK ] sdk sdl sdm Applying ktune sysctl settings: /etc/ktune.d/tunedadm.conf: [ OK ] Calling '/etc/ktune.d/tunedadm.sh start': [ OK ] Applying sysctl settings from /etc/sysctl.d/99-chef-attributes.conf Applying sysctl settings from /etc/sysctl.conf Starting tuned: [ OK ] 

 open("/proc/18679/cmdline", O_RDONLY) = 5 read(5, 

 Adaptec 71605