我们刚刚收到两台全新的Supermicro服务器1028U-TN10RT +，带有10个NVMe插槽，其中两个带有Intel DC P3600 800GB硬盘。
我们非常希望testing驱动器的性能，因为这个规范有很好的读取性能（高达2.6gb / s）和写入性能（高达1gb / s）。 我们把这两个驱动器放在软件RAID 1configuration中，因为这是我们想要在生产中使用的。 我们使用FIO运行testing，结果有些混乱。
完整的结果如下，但是回顾一下：RAID1arrays中的两个驱动器实现了〜550MB / s的随机写入速度（这是更好的运行之一），其中单个驱动器（无RAID）可以以〜920MB /秒。
使用软件RAID有多less开销？ 有一些额外的调整，我们可以做？
系统有128GB内存，正在运行一个CentOS 7.1，内核版本升级到4.2.4。
fio --name=randwrite --ioengine=libaio --iodepth=64 --rw=randwrite \ --bs=64k --direct=1 --size=32G --numjobs=8 --runtime=240 \ --group_reporting 单个驱动器xfs文件系统上的结果直接安装：
randwrite: (groupid=0, jobs=8): err= 0: pid=9307: Tue Oct 27 14:36:35 2015 write: io=217971MB, bw=929843KB/s, iops=14528, runt=240043msec slat (usec): min=5, max=933, avg=24.10, stdev= 9.29 clat (usec): min=32, max=135283, avg=35212.65, stdev=27746.71 lat (usec): min=49, max=135300, avg=35237.02, stdev=27746.76 clat percentiles (usec): | 1.00th=[ 215], 5.00th=[ 2224], 10.00th=[ 5600], 20.00th=[12992], | 30.00th=[16768], 40.00th=[19328], 50.00th=[23168], 60.00th=[33536], | 70.00th=[47872], 80.00th=[63232], 90.00th=[79360], 95.00th=[88576], | 99.00th=[102912], 99.50th=[107008], 99.90th=[116224], 99.95th=[119296], | 99.99th=[125440] bw (KB /s): min=42411, max=298624, per=12.51%, avg=116326.24, stdev=24050.53 lat (usec) : 50=0.01%, 100=0.27%, 250=0.87%, 500=0.77%, 750=0.55% lat (usec) : 1000=0.47% lat (msec) : 2=1.67%, 4=3.43%, 10=7.17%, 20=27.37%, 50=28.86% lat (msec) : 100=26.99%, 250=1.55% cpu : usr=1.75%, sys=4.98%, ctx=3056950, majf=0, minf=56673 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0% issued : total=r=0/w=3487535/d=0, short=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=64 Run status group 0 (all jobs): WRITE: io=217971MB, aggrb=929842KB/s, minb=929842KB/s, maxb=929842KB/s, mint=240043msec, maxt=240043msec Disk stats (read/write): nvme2n1: ios=0/4691372, merge=0/0, ticks=0/154695600, in_queue=155446639, util=100.00%
结果使用md时，RAID 1：
randwrite: (groupid=0, jobs=8): err= 0: pid=8553: Tue Oct 27 14:32:03 2015 write: io=130141MB, bw=555110KB/s, iops=8673, runt=240069msec slat (usec): min=20, max=349051, avg=130.51, stdev=2000.03 clat (usec): min=59, max=912669, avg=58782.87, stdev=50750.42 lat (usec): min=95, max=927440, avg=58913.81, stdev=51010.14 clat percentiles (usec): | 1.00th=[ 668], 5.00th=[ 3472], 10.00th=[ 8512], 20.00th=[21888], | 30.00th=[32640], 40.00th=[41728], 50.00th=[48896], 60.00th=[58112], | 70.00th=[71168], 80.00th=[86528], 90.00th=[114176], 95.00th=[142336], | 99.00th=[216064], 99.50th=[250880], 99.90th=[577536], 99.95th=[716800], | 99.99th=[872448] bw (KB /s): min= 70, max=175104, per=12.56%, avg=69708.68, stdev=20589.85 lat (usec) : 100=0.02%, 250=0.29%, 500=0.43%, 750=0.38%, 1000=0.36% lat (msec) : 2=1.22%, 4=2.98%, 10=5.56%, 20=7.47%, 50=32.45% lat (msec) : 100=34.50%, 250=13.81%, 500=0.39%, 750=0.08%, 1000=0.05% cpu : usr=1.28%, sys=6.46%, ctx=1727469, majf=0, minf=69488 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0% issued : total=r=0/w=2082262/d=0, short=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=64 Run status group 0 (all jobs): WRITE: io=130141MB, aggrb=555110KB/s, minb=555110KB/s, maxb=555110KB/s, mint=240069msec, maxt=240069msec Disk stats (read/write): md0: ios=0/2615652, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=11136/2630386, aggrmerge=0/0, aggrticks=10763/72152582, aggrin_queue=72527830, aggrutil=99.40% nvme0n1: ios=22273/2619265, merge=0/0, ticks=21526/14920779, in_queue=14979917, util=49.15% nvme1n1: ios=0/2641508, merge=0/0, ticks=0/129384385, in_queue=130075743, util=99.40%
mdadm –detail / dev / md0
/dev/md0: Version : 1.2 Creation Time : Tue Oct 27 13:12:34 2015 Raid Level : raid1 Array Size : 781278208 (745.08 GiB 800.03 GB) Used Dev Size : 781278208 (745.08 GiB 800.03 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Tue Oct 27 14:54:24 2015 State : clean Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Name : localhost.localdomain:0 (local to host localhost.localdomain) UUID : cf2ce291:0c52f361:bc40dffa:918595d9 Events : 706 Number Major Minor RaidDevice State 0 259 3 0 active sync /dev/nvme0n1p1 1 259 1 1 active sync /dev/nvme1n1p1
这可能是所使用的内部写意图位图的副作用。 使用mdadm <dev> --grow --bitmap=none将其删除，然后重新尝试使用fio 。
无论如何，我强烈build议您不要在没有启用位图的arrays的情况下进入生产阶段，因为崩溃/断电将迫使arrays执行完整的每字节字节扫描/比较。 写意图位图将保证更快的恢复。