Raid 10 – 很慢

我有软件RAID 10设置,它已经工作了好几个月。

当我做了一个快速的高清速度testing:

dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync 

这是永远的,我不得不终止它,结果是:

1073741824字节(1.1 GB)复制,151.27 s,7.1 MB / s

是什么造成这个?

 [root@host ~]# cat /proc/mdstat Personalities : [raid1] [raid10] md2 : active raid1 sdc1[4] sdd1[3] sdb1[1] sda1[0] 204788 blocks super 1.0 [4/4] [UUUU] md127 : active raid10 sdc4[4] sdb4[1] sdd4[3] sda4[0] 1915357184 blocks super 1.2 256K chunks 2 near-copies [4/4] [UUUU] md1 : active raid1 sdc3[4] sdb3[1] sdd3[3] sda3[0] 8387576 blocks super 1.1 [4/4] [UUUU] md0 : active raid1 sdc2[4] sda2[0] sdb2[1] sdd2[3] 10484668 blocks super 1.1 [4/4] [UUUU] bitmap: 1/1 pages [4KB], 65536KB chunk 

运行时间:

 [root@host ~]# uptime 18:50:28 up 105 days, 11:34, 1 user, load average: 0.04, 0.05, 0.00 

记忆

 [root@host ~]# free -m total used free shared buffers cached Mem: 15893 15767 125 0 461 14166 -/+ buffers/cache: 1139 14753 Swap: 8190 9 8181 

服务器规格:

至强E3-1230

16GB DDR-3 ECC

4 x 1TB 7200 RPM SATA(软件RAID)

我注意到当我做fdisk -l命令 – 得到一堆无效的分区,是不是有什么关系呢? 如果是的话如何解决?

 [root@host ~]# fdisk -l Disk /dev/sda: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x000d1a79 Device Boot Start End Blocks Id System /dev/sda1 * 1 26 204800 fd Linux raid autodetect Partition 1 does not end on cylinder boundary. /dev/sda2 26 1332 10485760 fd Linux raid autodetect /dev/sda3 1332 2376 8388608 fd Linux raid autodetect /dev/sda4 2376 121601 957679840+ fd Linux raid autodetect Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x000303b7 Device Boot Start End Blocks Id System /dev/sdb1 * 1 26 204800 fd Linux raid autodetect Partition 1 does not end on cylinder boundary. /dev/sdb2 26 1332 10485760 fd Linux raid autodetect /dev/sdb3 1332 2376 8388608 fd Linux raid autodetect /dev/sdb4 2376 121601 957679840+ fd Linux raid autodetect Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdc1 * 1 26 204800 fd Linux raid autodetect Partition 1 does not end on cylinder boundary. /dev/sdc2 26 1332 10485760 fd Linux raid autodetect /dev/sdc3 1332 2376 8388608 fd Linux raid autodetect /dev/sdc4 2376 121601 957679840+ fd Linux raid autodetect Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x0006436c Device Boot Start End Blocks Id System /dev/sdd1 * 1 26 204800 fd Linux raid autodetect Partition 1 does not end on cylinder boundary. /dev/sdd2 26 1332 10485760 fd Linux raid autodetect /dev/sdd3 1332 2376 8388608 fd Linux raid autodetect /dev/sdd4 2376 121601 957679840+ fd Linux raid autodetect Disk /dev/md0: 10.7 GB, 10736300032 bytes 2 heads, 4 sectors/track, 2621167 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/md0 doesn't contain a valid partition table Disk /dev/md1: 8588 MB, 8588877824 bytes 2 heads, 4 sectors/track, 2096894 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/md1 doesn't contain a valid partition table Disk /dev/md127: 1961.3 GB, 1961325756416 bytes 2 heads, 4 sectors/track, 478839296 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 524288 bytes Disk identifier: 0x00000000 Disk /dev/md127 doesn't contain a valid partition table Disk /dev/md2: 209 MB, 209702912 bytes 2 heads, 4 sectors/track, 51197 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/md2 doesn't contain a valid partition table 

我可以想到这样一个结果的几个潜在的原因:

  1. 太多的fdatasync()操作:在这种情况下,吞吐量受限于硬盘在特定时间范围内可执行的事务数量。 对于每个表面具有单头的传统旋转介质,绝对最大值由旋转速度确定。 例如对于64K的数据块,你只会得到

     7200rpm * 64K/rotation = 7,680 K/sec 

    我怀疑,由于RAID 1 + 0的语义,每写只会碰到一个单一的镜像,因为dd块的大小小于arrays块的大小。 这会将arrays的性能限制为单个镜像的性能,这对于单个驱动器来说是写入的。

    在我的系统dd与这些选项只执行一个单一的fdatasync()调用之前退出 – 也许你的每块一个? 在strace下运行dd会告诉你是否是这种情况。

  2. 装有sync数据选项的文件系统也会显示类似的行为。

  3. 硬件问题 – 由于电缆故障和硬盘故障,我经常看到这样的延迟。 我build议您仔细阅读系统日志并在所有驱动器上运行smartctl 。 读取testing也是有用的,因为它也应该受到硬件问题的影响 – 你的arrays如何读取?