我有一个RAID控制器的服务器，通过SAS连接到JBOD。 JBOD中装有16个3TB的SAS-HDD（一个8个，其他8个，相同的rpm）。 我configuration了3个带有RAID-5的逻辑驱动器，每个都有5个物理硬盘（+1作为热备份）。
现在来testing一下，我运行下面的脚本：
for i in 1 10 50 100 1000 do for j in abc do dd if=/dev/zero of=/dev/sd$j bs=1G count=$i done done 每次我运行这个脚本，它运行罚款= 1,10和50。
但是用100G，写在虚拟驱动器随机崩溃。 意思是，有时在/ dev / sda上写入没有和错误，但是崩溃了/ dev / sdb。 有时在/ dev / sda和/ dev / sdb上完成写入，但是在/ dev / sdc上写入失败。
我怀疑我的RAID卡可能有故障，因为我已经通过直接将它们连接到服务器并在完整的3TB上运行“dd”来单独testing我的硬盘。
你们有什么build议？
编辑 ：
服务器主板制造商/型号 ：SuperMicro X8DTH-6F
RAID控制器 ：带有BBU的LSI MegaRaid SAS 9285-8e
JBOD ：SuperMicro JBOD SC836E26-R1200B
SAS硬盘 ：3.5“，6Gbps SAS，3TB，7200rpm（8x – 希捷ST3000NM0023,8x – 日立Ultrastar 7K3000
操作系统 ：科学Linux 6.3
JBOD通过6Gbps SAS电缆连接到RAID控制器。
编辑2 ：这是来自/ var / log / messages的文本：
May 1 18:14:41 fileserver udevd[875]: worker [4083] unexpectedly returned with status 0x0100 May 1 18:14:41 fileserver udevd[875]: worker [4083] failed while handling '/devices/pci0000:00/0000:00:03.0/0000:08:00.0/host0/target0:2:0/0:2:0:0/block/sda' May 1 18:14:43 fileserver kernel: megasas: Found FW in FAULT state, will reset adapter. May 1 18:14:43 fileserver kernel: megaraid_sas: resetting fusion adapter. May 1 18:15:17 fileserver kernel: INFO: task dd:4144 blocked for more than 120 seconds. May 1 18:15:17 fileserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 1 18:15:17 fileserver kernel: dd D 000000000000000d 0 4144 4132 0x00000004 May 1 18:15:17 fileserver kernel: ffff88060c513bf8 0000000000000082 0000000000000000 ffffea0015a3a8c0 May 1 18:15:17 fileserver kernel: ffff88062d837938 ffff88062d837848 ffff88062ce06ea0 ffff88062ce06ea0 May 1 18:15:17 fileserver kernel: ffff8806133385f8 ffff88060c513fd8 000000000000fb88 ffff8806133385f8 May 1 18:15:17 fileserver kernel: Call Trace: May 1 18:15:17 fileserver kernel: [<ffffffff811140c0>] ? sync_page+0x0/0x50 May 1 18:15:17 fileserver kernel: [<ffffffff814fdfc3>] io_schedule+0x73/0xc0 May 1 18:15:17 fileserver kernel: [<ffffffff811140fd>] sync_page+0x3d/0x50 May 1 18:15:17 fileserver kernel: [<ffffffff814fe97f>] __wait_on_bit+0x5f/0x90 May 1 18:15:17 fileserver kernel: [<ffffffff81114333>] wait_on_page_bit+0x73/0x80 May 1 18:15:17 fileserver kernel: [<ffffffff81092110>] ? wake_bit_function+0x0/0x50 May 1 18:15:17 fileserver kernel: [<ffffffff8112a835>] ? pagevec_lookup_tag+0x25/0x40 May 1 18:15:17 fileserver kernel: [<ffffffff811147ab>] wait_on_page_writeback_range+0xfb/0x190 May 1 18:15:17 fileserver kernel: [<ffffffff8111486f>] filemap_fdatawait+0x2f/0x40 May 1 18:15:17 fileserver kernel: [<ffffffff81114e94>] filemap_write_and_wait+0x44/0x60 May 1 18:15:17 fileserver kernel: [<ffffffff811b4634>] __sync_blockdev+0x24/0x50 May 1 18:15:17 fileserver kernel: [<ffffffff811b4673>] sync_blockdev+0x13/0x20 May 1 18:15:17 fileserver kernel: [<ffffffff811b47f8>] __blkdev_put+0x178/0x1b0 May 1 18:15:17 fileserver kernel: [<ffffffff811b9403>] ? fsnotify+0x113/0x160 May 1 18:15:17 fileserver kernel: [<ffffffff811b4840>] blkdev_put+0x10/0x20 May 1 18:15:17 fileserver kernel: [<ffffffff811b4883>] blkdev_close+0x33/0x60 May 1 18:15:17 fileserver kernel: [<ffffffff8117c935>] __fput+0xf5/0x210 May 1 18:15:17 fileserver kernel: [<ffffffff8117ca75>] fput+0x25/0x30 May 1 18:15:17 fileserver kernel: [<ffffffff8117849d>] filp_close+0x5d/0x90 May 1 18:15:17 fileserver kernel: [<ffffffff81178575>] sys_close+0xa5/0x100 May 1 18:15:17 fileserver kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b May 1 18:15:17 fileserver kernel: INFO: task scsi_id:4145 blocked for more than 120 seconds. May 1 18:15:17 fileserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 1 18:15:17 fileserver kernel: scsi_id D 0000000000000013 0 4145 1 0x00000000 May 1 18:15:17 fileserver kernel: ffff880c1a145c18 0000000000000086 0000000000000000 ffff880c00000002 May 1 18:15:17 fileserver kernel: ffff880c1a145be8 00007f9200000066 00000000007fffff ffffffff81fc7990 May 1 18:15:17 fileserver kernel: ffff880c2ee3b098 ffff880c1a145fd8 000000000000fb88 ffff880c2ee3b098 May 1 18:15:17 fileserver kernel: Call Trace: May 1 18:15:17 fileserver kernel: [<ffffffff814fee8e>] __mutex_lock_slowpath+0x13e/0x180 May 1 18:15:17 fileserver kernel: [<ffffffff8125ee50>] ? exact_match+0x0/0x10 May 1 18:15:17 fileserver kernel: [<ffffffff814fed2b>] mutex_lock+0x2b/0x50 May 1 18:15:17 fileserver kernel: [<ffffffff811b4958>] __blkdev_get+0x68/0x3c0 May 1 18:15:17 fileserver kernel: [<ffffffff811b4cd0>] ? blkdev_open+0x0/0xc0 May 1 18:15:17 fileserver kernel: [<ffffffff811b4cc0>] blkdev_get+0x10/0x20 May 1 18:15:17 fileserver kernel: [<ffffffff811b4d41>] blkdev_open+0x71/0xc0 May 1 18:15:17 fileserver kernel: [<ffffffff8117888a>] __dentry_open+0x10a/0x360 May 1 18:15:17 fileserver kernel: [<ffffffff8121c262>] ? selinux_inode_permission+0x72/0xb0 May 1 18:15:17 fileserver kernel: [<ffffffff8121429f>] ? security_inode_permission+0x1f/0x30 May 1 18:15:17 fileserver kernel: [<ffffffff81178bf4>] nameidata_to_filp+0x54/0x70 May 1 18:15:17 fileserver kernel: [<ffffffff8118c100>] do_filp_open+0x6c0/0xd60 May 1 18:15:17 fileserver kernel: [<ffffffff8104452c>] ? __do_page_fault+0x1ec/0x480 May 1 18:15:17 fileserver kernel: [<ffffffff812718b1>] ? cpumask_any_but+0x31/0x50 May 1 18:15:17 fileserver kernel: [<ffffffff811449a0>] ? unmap_region+0x110/0x130 May 1 18:15:17 fileserver kernel: [<ffffffff81198182>] ? alloc_fd+0x92/0x160 May 1 18:15:17 fileserver kernel: [<ffffffff81178639>] do_sys_open+0x69/0x140 May 1 18:15:17 fileserver kernel: [<ffffffff81178750>] sys_open+0x20/0x30 May 1 18:15:17 fileserver kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b May 1 18:15:17 fileserver kernel: INFO: task fdisk:4176 blocked for more than 120 seconds. May 1 18:15:17 fileserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 1 18:15:17 fileserver kernel: fdisk D 0000000000000013 0 4176 4147 0x00000004 May 1 18:15:17 fileserver kernel: ffff880c2f56dc18 0000000000000082 0000000000000000 ffff880c30802078 May 1 18:15:17 fileserver kernel: ffff880c2f56dbe8 ffff880c00000721 00000000007fffff ffffffff81fc8368 May 1 18:15:17 fileserver kernel: ffff880c1d6af058 ffff880c2f56dfd8 000000000000fb88 ffff880c1d6af058 May 1 18:15:17 fileserver kernel: Call Trace: May 1 18:15:17 fileserver kernel: [<ffffffff814fee8e>] __mutex_lock_slowpath+0x13e/0x180 May 1 18:15:17 fileserver kernel: [<ffffffff8125ee50>] ? exact_match+0x0/0x10 May 1 18:15:17 fileserver kernel: [<ffffffff814fed2b>] mutex_lock+0x2b/0x50 May 1 18:15:17 fileserver kernel: [<ffffffff811b4958>] __blkdev_get+0x68/0x3c0 May 1 18:15:17 fileserver kernel: [<ffffffff811b4cd0>] ? blkdev_open+0x0/0xc0 May 1 18:15:17 fileserver kernel: [<ffffffff811b4cc0>] blkdev_get+0x10/0x20 May 1 18:15:17 fileserver kernel: [<ffffffff811b4d41>] blkdev_open+0x71/0xc0 May 1 18:15:17 fileserver kernel: [<ffffffff8117888a>] __dentry_open+0x10a/0x360 May 1 18:15:17 fileserver kernel: [<ffffffff8121c262>] ? selinux_inode_permission+0x72/0xb0 May 1 18:15:17 fileserver kernel: [<ffffffff8121429f>] ? security_inode_permission+0x1f/0x30 May 1 18:15:17 fileserver kernel: [<ffffffff81178bf4>] nameidata_to_filp+0x54/0x70 May 1 18:15:17 fileserver kernel: [<ffffffff8118c100>] do_filp_open+0x6c0/0xd60 May 1 18:15:17 fileserver kernel: [<ffffffff8104452c>] ? __do_page_fault+0x1ec/0x480 May 1 18:15:17 fileserver kernel: [<ffffffff811e06c5>] ? pde_users_dec+0x25/0x60 May 1 18:15:17 fileserver kernel: [<ffffffff81198182>] ? alloc_fd+0x92/0x160 May 1 18:15:17 fileserver kernel: [<ffffffff81178639>] do_sys_open+0x69/0x140 May 1 18:15:17 fileserver kernel: [<ffffffff81178750>] sys_open+0x20/0x30 May 1 18:15:17 fileserver kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b May 1 18:16:27 fileserver kernel: megaraid_sas: Diag reset adapter never cleared! May 1 18:17:17 fileserver kernel: INFO: task dd:4144 blocked for more than 120 seconds. May 1 18:17:17 fileserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 1 18:17:17 fileserver kernel: dd D 000000000000000d 0 4144 4132 0x00000004 May 1 18:17:17 fileserver kernel: ffff88060c513bf8 0000000000000082 0000000000000000 ffffea0015a3a8c0 May 1 18:17:17 fileserver kernel: ffff88062d837938 ffff88062d837848 ffff88062ce06ea0 ffff88062ce06ea0 May 1 18:17:17 fileserver kernel: ffff8806133385f8 ffff88060c513fd8 000000000000fb88 ffff8806133385f8 May 1 18:17:17 fileserver kernel: Call Trace: May 1 18:17:17 fileserver kernel: [<ffffffff811140c0>] ? sync_page+0x0/0x50 May 1 18:17:17 fileserver kernel: [<ffffffff814fdfc3>] io_schedule+0x73/0xc0 May 1 18:17:17 fileserver kernel: [<ffffffff811140fd>] sync_page+0x3d/0x50 May 1 18:17:17 fileserver kernel: [<ffffffff814fe97f>] __wait_on_bit+0x5f/0x90 May 1 18:17:17 fileserver kernel: [<ffffffff81114333>] wait_on_page_bit+0x73/0x80 May 1 18:17:17 fileserver kernel: [<ffffffff81092110>] ? wake_bit_function+0x0/0x50 May 1 18:17:17 fileserver kernel: [<ffffffff8112a835>] ? pagevec_lookup_tag+0x25/0x40 May 1 18:17:17 fileserver kernel: [<ffffffff811147ab>] wait_on_page_writeback_range+0xfb/0x190 May 1 18:17:17 fileserver kernel: [<ffffffff8111486f>] filemap_fdatawait+0x2f/0x40 May 1 18:17:17 fileserver kernel: [<ffffffff81114e94>] filemap_write_and_wait+0x44/0x60 May 1 18:17:17 fileserver kernel: [<ffffffff811b4634>] __sync_blockdev+0x24/0x50 May 1 18:17:17 fileserver kernel: [<ffffffff811b4673>] sync_blockdev+0x13/0x20 May 1 18:17:17 fileserver kernel: [<ffffffff811b47f8>] __blkdev_put+0x178/0x1b0 May 1 18:17:17 fileserver kernel: [<ffffffff811b9403>] ? fsnotify+0x113/0x160 May 1 18:17:17 fileserver kernel: [<ffffffff811b4840>] blkdev_put+0x10/0x20 May 1 18:17:17 fileserver kernel: [<ffffffff811b4883>] blkdev_close+0x33/0x60 May 1 18:17:17 fileserver kernel: [<ffffffff8117c935>] __fput+0xf5/0x210 May 1 18:17:17 fileserver kernel: [<ffffffff8117ca75>] fput+0x25/0x30 May 1 18:17:17 fileserver kernel: [<ffffffff8117849d>] filp_close+0x5d/0x90 May 1 18:17:17 fileserver kernel: [<ffffffff81178575>] sys_close+0xa5/0x100 May 1 18:17:17 fileserver kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b May 1 18:17:17 fileserver kernel: INFO: task scsi_id:4145 blocked for more than 120 seconds. May 1 18:17:17 fileserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 1 18:17:17 fileserver kernel: scsi_id D 0000000000000013 0 4145 1 0x00000000 May 1 18:17:17 fileserver kernel: ffff880c1a145c18 0000000000000086 0000000000000000 ffff880c00000002 May 1 18:17:17 fileserver kernel: ffff880c1a145be8 00007f9200000066 00000000007fffff ffffffff81fc7990 May 1 18:17:17 fileserver kernel: ffff880c2ee3b098 ffff880c1a145fd8 000000000000fb88 ffff880c2ee3b098 May 1 18:17:17 fileserver kernel: Call Trace: May 1 18:17:17 fileserver kernel: [<ffffffff814fee8e>] __mutex_lock_slowpath+0x13e/0x180 May 1 18:17:17 fileserver kernel: [<ffffffff8125ee50>] ? exact_match+0x0/0x10 May 1 18:17:17 fileserver kernel: [<ffffffff814fed2b>] mutex_lock+0x2b/0x50 May 1 18:17:17 fileserver kernel: [<ffffffff811b4958>] __blkdev_get+0x68/0x3c0 May 1 18:17:17 fileserver kernel: [<ffffffff811b4cd0>] ? blkdev_open+0x0/0xc0 May 1 18:17:17 fileserver kernel: [<ffffffff811b4cc0>] blkdev_get+0x10/0x20 May 1 18:17:17 fileserver kernel: [<ffffffff811b4d41>] blkdev_open+0x71/0xc0 May 1 18:17:17 fileserver kernel: [<ffffffff8117888a>] __dentry_open+0x10a/0x360 May 1 18:17:17 fileserver kernel: [<ffffffff8121c262>] ? selinux_inode_permission+0x72/0xb0 May 1 18:17:17 fileserver kernel: [<ffffffff8121429f>] ? security_inode_permission+0x1f/0x30 May 1 18:17:17 fileserver kernel: [<ffffffff81178bf4>] nameidata_to_filp+0x54/0x70 May 1 18:17:17 fileserver kernel: [<ffffffff8118c100>] do_filp_open+0x6c0/0xd60 May 1 18:17:17 fileserver kernel: [<ffffffff8104452c>] ? __do_page_fault+0x1ec/0x480 May 1 18:17:17 fileserver kernel: [<ffffffff812718b1>] ? cpumask_any_but+0x31/0x50 May 1 18:17:17 fileserver kernel: [<ffffffff811449a0>] ? unmap_region+0x110/0x130 May 1 18:17:17 fileserver kernel: [<ffffffff81198182>] ? alloc_fd+0x92/0x160 May 1 18:17:17 fileserver kernel: [<ffffffff81178639>] do_sys_open+0x69/0x140 May 1 18:17:17 fileserver kernel: [<ffffffff81178750>] sys_open+0x20/0x30 May 1 18:17:17 fileserver kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b May 1 18:17:17 fileserver kernel: INFO: task fdisk:4176 blocked for more than 120 seconds. May 1 18:17:17 fileserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 1 18:17:17 fileserver kernel: fdisk D 0000000000000013 0 4176 4147 0x00000004 May 1 18:17:17 fileserver kernel: ffff880c2f56dc18 0000000000000082 0000000000000000 ffff880c30802078 May 1 18:17:17 fileserver kernel: ffff880c2f56dbe8 ffff880c00000721 00000000007fffff ffffffff81fc8368 May 1 18:17:17 fileserver kernel: ffff880c1d6af058 ffff880c2f56dfd8 000000000000fb88 ffff880c1d6af058 May 1 18:17:17 fileserver kernel: Call Trace: May 1 18:17:17 fileserver kernel: [<ffffffff814fee8e>] __mutex_lock_slowpath+0x13e/0x180 May 1 18:17:17 fileserver kernel: [<ffffffff8125ee50>] ? exact_match+0x0/0x10 May 1 18:17:17 fileserver kernel: [<ffffffff814fed2b>] mutex_lock+0x2b/0x50 May 1 18:17:17 fileserver kernel: [<ffffffff811b4958>] __blkdev_get+0x68/0x3c0 May 1 18:17:17 fileserver kernel: [<ffffffff811b4cd0>] ? blkdev_open+0x0/0xc0 May 1 18:17:17 fileserver kernel: [<ffffffff811b4cc0>] blkdev_get+0x10/0x20 May 1 18:17:17 fileserver kernel: [<ffffffff811b4d41>] blkdev_open+0x71/0xc0 May 1 18:17:17 fileserver kernel: [<ffffffff8117888a>] __dentry_open+0x10a/0x360 May 1 18:17:17 fileserver kernel: [<ffffffff8121c262>] ? selinux_inode_permission+0x72/0xb0 May 1 18:17:17 fileserver kernel: [<ffffffff8121429f>] ? security_inode_permission+0x1f/0x30 May 1 18:17:17 fileserver kernel: [<ffffffff81178bf4>] nameidata_to_filp+0x54/0x70 May 1 18:17:17 fileserver kernel: [<ffffffff8118c100>] do_filp_open+0x6c0/0xd60 May 1 18:17:17 fileserver kernel: [<ffffffff8104452c>] ? __do_page_fault+0x1ec/0x480 May 1 18:17:17 fileserver kernel: [<ffffffff811e06c5>] ? pde_users_dec+0x25/0x60 May 1 18:17:17 fileserver kernel: [<ffffffff81198182>] ? alloc_fd+0x92/0x160 May 1 18:17:17 fileserver kernel: [<ffffffff81178639>] do_sys_open+0x69/0x140 May 1 18:17:17 fileserver kernel: [<ffffffff81178750>] sys_open+0x20/0x30 May 1 18:17:17 fileserver kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b May 1 18:18:11 fileserver kernel: megaraid_sas: Diag reset adapter never cleared! ... ...
我无法做任何事情，它呼救！
结果发现，Supermicro服务器上的Scientific Linux 6.3在PCIe方面存在一些问题。 一位朋友build议将以下两个选项添加到grub中：
用这些选项启动后，一切开始正常工作。
有什么想法吗？