好的 – 有人在问我关于你的问题，所以我启动了一个VM来潜入应该预期的行为。 我会在一分钟之内知道是什么让我烦恼的。 首先让我说这个：

在尝试任何事情之前备份这些驱动器！

你可能已经做了超过重新同步的损害。 你能说清楚你的意思吗？

根据build议，我没有清理超级块，并重新创build了–assume-clean选项的数组，但没有运气。

如果你运行了mdadm --misc --zero-superblock ，那么你应该没问题。

无论如何，在做任何可能再写入这些磁盘的东西之前，清理一些新的磁盘并抓取它们当前的确切图像。

 dd if=/dev/sdd of=/path/to/store/sdd.img 

这就是说，它看起来像存储在这些东西的数据是震惊韧性任性resyncs。 请继续阅读，有希望，这可能是我达到答案长度限制的那一天。

---

最好的案例情景

我把一个虚拟机放在一起来重新创build你的场景。 驱动器只有100 MB，所以我不会永远等待每个resync，但这应该是一个相当准确的表示，否则。

build立数组一般和默认尽可能 – 512K块，左对称布局，磁盘顺序..没有什么特别的。

 root@test:~# mdadm --create /dev/md0 --chunk=512 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. root@test:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md0 : active raid5 sdd1[3] sdc1[1] sdb1[0] 203776 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> 

到现在为止还挺好; 让我们创build一个文件系统，并在其上放置一些数据。

 root@test:~# mkfs.ext4 /dev/md0 mke2fs 1.41.14 (22-Dec-2010) Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) Stride=512 blocks, Stripe width=1024 blocks 51000 inodes, 203776 blocks 10188 blocks (5.00%) reserved for the super user First data block=1 Maximum filesystem blocks=67371008 25 block groups 8192 blocks per group, 8192 fragments per group 2040 inodes per group Superblock backups stored on blocks: 8193, 24577, 40961, 57345, 73729 Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 30 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. root@test:~# mkdir /mnt/raid5 root@test:~# mount /dev/md0 /mnt/raid5 root@test:~# echo "data" > /mnt/raid5/datafile root@test:~# dd if=/dev/urandom of=/mnt/raid5/randomdata count=10000 10000+0 records in 10000+0 records out 5120000 bytes (5.1 MB) copied, 0.706526 s, 7.2 MB/s root@test:~# sha1sum /mnt/raid5/randomdata 847685a5d42524e5b1d5484452a649e854b59064 /mnt/raid5/randomdata 

好。 我们有一个文件系统和一些数据（数据datafile “数据”，随机数据中包含5MB的随机数据，随机数据中包含SHA1哈希）; 让我们看看当我们重新创build时会发生什么。

 root@test:~# umount /mnt/raid5 root@test:~# mdadm --stop /dev/md0 mdadm: stopped /dev/md0 root@test:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] unused devices: <none> root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 21:07:06 2012 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 21:07:06 2012 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 21:07:06 2012 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid5 sdd1[2] sdc1[1] sdb1[0] 203776 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> 

重新同步很快用这些小磁盘完成，但确实发生了。 所以这是从前面的问题。 你的fdisk -l输出。 在md设备上没有分区表完全不是问题，这是预料之中的。 您的文件系统直接驻留在没有分区表的假模块设备上。

 root@test:~# fdisk -l ... Disk /dev/md1: 208 MB, 208666624 bytes 2 heads, 4 sectors/track, 50944 cylinders, total 407552 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000 Disk /dev/md1 doesn't contain a valid partition table 

是的，没有分区表。 但…

 root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) /dev/md1: clean, 12/51000 files, 12085/203776 blocks 

完全有效的文件系统，重新同步后。 所以这很好， 让我们来看看我们的数据文件：

 root@test:~# mount /dev/md1 /mnt/raid5/ root@test:~# cat /mnt/raid5/datafile data root@test:~# sha1sum /mnt/raid5/randomdata 847685a5d42524e5b1d5484452a649e854b59064 /mnt/raid5/randomdata 

固体 – 根本没有数据损坏！ 但这是完全相同的设置，所以两个RAID组之间没有任何映射。 在我们试图打破它之前，先放下这个东西。

 root@test:~# umount /mnt/raid5 root@test:~# mdadm --stop /dev/md1 

---

退一步

在我们试图打破这个之前，让我们来谈谈为什么它很难打破。 RAID 5通过使用奇偶校验块工作，该奇偶校验块保护与arrays中每个其他磁盘上的块大小相同的区域。 奇偶校验并不只是在一个特定的磁盘上，而是在磁盘上均匀旋转，以便在正常操作中更好地将读取负载分散到磁盘上。

计算奇偶校验的XOR操作如下所示：

 DISK1 DISK2 DISK3 DISK4 PARITY 1 0 1 1 = 1 0 0 1 1 = 0 1 1 1 1 = 0 

所以，奇偶校验分散在磁盘之间。

 DISK1 DISK2 DISK3 DISK4 DISK5 DATA DATA DATA DATA PARITY PARITY DATA DATA DATA DATA DATA PARITY DATA DATA DATA 

通常在更换死磁盘或丢失的磁盘时执行重新同步; 它也是在mdadm create上完成的，以确保磁盘上的数据与RAID的几何graphics应该是一致的。 在这种情况下，arrays规格中的最后一个磁盘是“同步到”的磁盘 – 其他磁盘上的所有现有数据都用于同步。

因此，“新”磁盘上的所有数据都被清除并重build; 要么从奇偶校验块中创build新的数据块，要么build立新的奇偶校验块。

最酷的是，这两件事的过程是完全相同的：对数据和其余磁盘进行XOR操作。 在这种情况下，重新同步过程可能在其布局中有一个特定的块应该是一个奇偶校验块，并且认为它正在build立一个新的奇偶校验块，实际上它正在重新创build一个旧的数据块。 所以即使它认为它正在build设这个：

 DISK1 DISK2 DISK3 DISK4 DISK5 PARITY DATA DATA DATA DATA DATA PARITY DATA DATA DATA DATA DATA PARITY DATA DATA 

…它可能只是从上面的布局重buildDISK5 。

所以，即使数组构build错误，数据也可能保持一致。

---

在作品中扔猴子

（不是扳手;整只猴子）

testing1：

让我们按照错误的顺序创build数组！ sdc ，然后sdd ，然后sdb ..

 root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --raid-devices=3 /dev/sdc1 /dev/sdd1 /dev/sdb1 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:06:34 2012 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:06:34 2012 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:06:34 2012 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid5 sdb1[3] sdd1[1] sdc1[0] 203776 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> 

好的，这一切都很好。 我们有文件系统吗？

 root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) fsck.ext4: Superblock invalid, trying backup blocks... fsck.ext4: Bad magic number in super-block while trying to open /dev/md1 The superblock could not be read or does not describe a correct ext2 filesystem. If the device is valid and it really contains an ext2 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock: e2fsck -b 8193 <device> 

不！ 这是为什么？ 因为当数据全部存在时，它的顺序是错误的; 那么曾经512KB的A，然后512KB的B，A，B等等现在已经被洗牌到了B，A，B，A。现在磁盘对文件系统检查器看起来很乱，不会运行。 mdadm --misc -D /dev/md1的输出给了我们更多的细节; 它看起来像这样：

 Number Major Minor RaidDevice State 0 8 33 0 active sync /dev/sdc1 1 8 49 1 active sync /dev/sdd1 3 8 17 2 active sync /dev/sdb1 

当它看起来像这样：

 Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1 3 8 49 2 active sync /dev/sdd1 

所以，这一切都很好。 我们用新的奇偶校验块重写了一大堆数据块。 现在用正确的顺序重新创build：

 root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:11:08 2012 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:11:08 2012 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:11:08 2012 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) /dev/md1: clean, 12/51000 files, 12085/203776 blocks 

整洁，还有一个文件系统！ 还有数据？

 root@test:~# mount /dev/md1 /mnt/raid5/ root@test:~# cat /mnt/raid5/datafile data root@test:~# sha1sum /mnt/raid5/randomdata 847685a5d42524e5b1d5484452a649e854b59064 /mnt/raid5/randomdata 

成功！

testing2

好吧，让我们改变块大小，看看是否让我们破碎。

 root@test:~# umount /mnt/raid5 root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 root@test:~# mdadm --create /dev/md1 --chunk=64 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:21:19 2012 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:21:19 2012 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:21:19 2012 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) fsck.ext4: Superblock invalid, trying backup blocks... fsck.ext4: Bad magic number in super-block while trying to open /dev/md1 The superblock could not be read or does not describe a correct ext2 filesystem. If the device is valid and it really contains an ext2 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock: e2fsck -b 8193 <device> 

是的，是的，当这样设置的时候就会被冲洗干净。 但是，我们能恢复吗？

 root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:21:51 2012 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:21:51 2012 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:21:51 2012 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) /dev/md1: clean, 12/51000 files, 12085/203776 blocks root@test:~# mount /dev/md1 /mnt/raid5/ root@test:~# cat /mnt/raid5/datafile data root@test:~# sha1sum /mnt/raid5/randomdata 847685a5d42524e5b1d5484452a649e854b59064 /mnt/raid5/randomdata 

再次成功！

testing3

这是我认为会杀死数据的一个 – 让我们做一个不同的布局algorithm！

 root@test:~# umount /mnt/raid5 root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --layout=right-asymmetric --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:32:34 2012 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:32:34 2012 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:32:34 2012 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid5 sdd1[3] sdc1[1] sdb1[0] 203776 blocks super 1.2 level 5, 512k chunk, algorithm 1 [3/3] [UUU] unused devices: <none> root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) fsck.ext4: Superblock invalid, trying backup blocks... Superblock has an invalid journal (inode 8). 

可怕和坏 – 它认为它发现了一些东西，想要做一些修复！ Ctrl + C ！

 Clear<y>? cancelled! fsck.ext4: Illegal inode number while checking ext3 journal for /dev/md1 

好吧，危机避免了。 让我们看看在使用错误的布局重新同步后，数据是否仍然完好：

 root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:33:02 2012 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:33:02 2012 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Sat Jan 7 23:33:02 2012 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) /dev/md1: clean, 12/51000 files, 12085/203776 blocks root@test:~# mount /dev/md1 /mnt/raid5/ root@test:~# cat /mnt/raid5/datafile data root@test:~# sha1sum /mnt/raid5/randomdata 847685a5d42524e5b1d5484452a649e854b59064 /mnt/raid5/randomdata 

成功！

testing4

我们也只是certificate这个超级块调零实际上并不危害：

 root@test:~# umount /mnt/raid5 root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 root@test:~# mdadm --misc --zero-superblock /dev/sdb1 /dev/sdc1 /dev/sdd1 root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) /dev/md1: clean, 12/51000 files, 12085/203776 blocks root@test:~# mount /dev/md1 /mnt/raid5/ root@test:~# cat /mnt/raid5/datafile data root@test:~# sha1sum /mnt/raid5/randomdata 847685a5d42524e5b1d5484452a649e854b59064 /mnt/raid5/randomdata 

对，没什么大不了的

testing5

让我们把我们所拥有的一切抛在脑后。 所有4个先前的testing结合起来。

• 错误的设备顺序
• 错误的块大小
• 错误的布局algorithm
• 归零超块（我们将在两个创作之间做这个）

向前！

 root@test:~# umount /mnt/raid5 root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 root@test:~# mdadm --misc --zero-superblock /dev/sdb1 /dev/sdc1 /dev/sdd1 root@test:~# mdadm --create /dev/md1 --chunk=64 --level=5 --raid-devices=3 --layout=right-symmetric /dev/sdc1 /dev/sdd1 /dev/sdb1 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid5 sdb1[3] sdd1[1] sdc1[0] 204672 blocks super 1.2 level 5, 64k chunk, algorithm 3 [3/3] [UUU] unused devices: <none> root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) fsck.ext4: Superblock invalid, trying backup blocks... fsck.ext4: Bad magic number in super-block while trying to open /dev/md1 The superblock could not be read or does not describe a correct ext2 filesystem. If the device is valid and it really contains an ext2 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock: e2fsck -b 8193 <device> root@test:~# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 

判决？

 root@test:~# mdadm --misc --zero-superblock /dev/sdb1 /dev/sdc1 /dev/sdd1 root@test:~# mdadm --create /dev/md1 --chunk=512 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. root@test:~# cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid5 sdd1[3] sdc1[1] sdb1[0] 203776 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> root@test:~# fsck.ext4 /dev/md1 e2fsck 1.41.14 (22-Dec-2010) /dev/md1: clean, 13/51000 files, 17085/203776 blocks root@test:~# mount /dev/md1 /mnt/raid5/ root@test:~# cat /mnt/raid5/datafile data root@test:~# sha1sum /mnt/raid5/randomdata 847685a5d42524e5b1d5484452a649e854b59064 /mnt/raid5/randomdata 

哇。

所以，看起来这些行为都没有以任何方式破坏数据。 坦率地说，我对这个结果感到非常惊讶。 我预计数据块大小变化会导致数据丢失的可能性较小，并且布局变化会造成一定的损失。 今天我学到了一些东西

---

所以..我如何获得我的数据？

尽可能多的关于旧系统的信息对您非常有帮助。 如果您知道文件系统types，那么您的/proc/mdstat有任何旧版本的驱动器顺序，algorithm，块大小和元数据版本信息。 你有没有设置mdadm的电子邮件警报？ 如果是这样，找一个老的; 如果没有，检查/var/spool/mail/root 。 检查你的~/.bash_history ，看看你的原始版本是否在那里。

所以，你应该做的事情清单：

1. 在做任何事之前用dd备份磁盘！
2. 尝试fsck当前活动的md – 您可能刚刚按照与以前相同的顺序进行构build。 如果你知道文件系统types，这是有帮助的; 使用特定的fsck工具。 如果任何工具提供修复任何东西，不要让他们，除非你确定他们已经find了有效的文件系统！ 如果fsck提供了修复某些东西的方法，请不要犹豫，留下一个问题，询问它是真正帮助还是只是要保留数据。
3. 尝试使用不同的参数构build数组。 如果你有一个旧的/proc/mdstat ，那么你可以模仿它显示的东西; 如果不是的话，那么你就是在黑暗中 – 尝试所有不同的驱动器命令是合理的，但检查每一个可能的顺序每个可能的块大小是徒劳的。 对于每一个， fsck它看看你是否有任何希望。

那就是这样 对于小说不好意思，如果您有任何问题，请随时发表评论，祝您好运！

脚注：22000字以下; 8K +害羞的长度限制

如果幸运的话，您可以通过使用可以读取损坏的RAID-5arrays的恢复软件来恢复文件。 零假设恢复是我曾经有过的成功。

但是，我不确定创build一个新数组的过程是否已经结束并销毁了所有的数据，所以这可能是最后的机会。

我有一个类似的问题：
在一个软件RAID5arrays发生故障之后，我解雇了mdadm --create而不给它--assume-clean ，并且无法再安装这个arrays。 经过两周的挖掘，我终于恢复了所有数据。 我希望下面的程序能节省一些时间。

长话短说

问题是由于mdadm --create在两个方面创build了一个与原来不同的新数组：

• 不同的分区顺序
• 不同的RAID数据偏移

正如Shane Madden的精彩回答所显示的，在大多数情况下， mdadm --create不会破坏数据！ find分区顺序和数据偏移后，我可以恢复数组并从中提取所有数据。

先决条件

我没有RAID超级块的备份，所以我只知道它是安装Xubuntu 12.04.0时创build的8个分区上的RAID5arrays。 它有一个ext4文件系统。 另一个重要的知识是存储在RAIDarrays上的文件副本。

工具

Xubuntu 12.04.1 live CD被用来做所有的工作。 根据您的情况，您可能需要以下一些工具：

允许指定数据偏移量的mdadm版本

 sudo apt-get install binutils-dev git git clone -b data_offset git://neil.brown.name/mdadm cd mdadm make 

bgrep – search二进制数据

 curl -L 'https://github.com/tmbinc/bgrep/raw/master/bgrep.c' | gcc -O2 -xc -o bgrep - 

hexdump，e2fsck，mount和一个hex计算器 – 从回购标准工具

从完整备份开始

设备文件的命名（例如/dev/sda2 /dev/sdb2等）并不是永久性的，所以最好写下你的驱动器的序列号

 sudo hdparm -I /dev/sda 

然后挂上一个外部硬盘，并像这样备份RAIDarrays的每个分区：

 sudo dd if=/dev/sda2 bs=4M | gzip > serial-number.gz 

确定原始RAID5布局

这里描述了各种布局： http : //www.accs.com/p_and_p/RAID/LinuxRAID.html
要find原始数组上的数据条是如何组织的，你需要一个你知道存储在数组中的随机文件的副本。 mdadm当前使用的默认块大小是512KB。 对于N个分区的数组，您需要一个至less为（N + 1）* 512KB的文件。 一个JPEG或video是好的，因为它提供了相对独特的二进制数据子串。 假设我们的文件名为picture.jpg 。 我们从100k开始，在N + 1个位置读取32个字节的数据，并增加512k：

 hexdump -n32 -s100k -v -e '/1 "%02X"' picture.jpg ; echo DA1DC4D616B1C71079624CDC36E3D40E7B1CFF00857C663687B6C4464D6C77D2 hexdump -n32 -s612k -v -e '/1 "%02X"' picture.jpg ; echo AB9DDDBBB05CA915EE2289E59A116B02A26C82C8A8033DD8FA6D06A84B6501B7 hexdump -n32 -s1124k -v -e '/1 "%02X"' picture.jpg ; echo BC31A8DC791ACDA4FA3E9D3406D5639619576AEE2E08C03C9EF5E23F0A7C5CBA ... 

然后，我们在所有的原始分区上search所有这些字节的出现，所以总共（N + 1）* N个命令如下所示：

 sudo ./bgrep DA1DC4D616B1C71079624CDC36E3D40E7B1CFF00857C663687B6C4464D6C77D2 /dev/sda2 sudo ./bgrep DA1DC4D616B1C71079624CDC36E3D40E7B1CFF00857C663687B6C4464D6C77D2 /dev/sdb2 ... sudo ./bgrep DA1DC4D616B1C71079624CDC36E3D40E7B1CFF00857C663687B6C4464D6C77D2 /dev/sdh2 /dev/sdh2: 52a7ff000 sudo ./bgrep AB9DDDBBB05CA915EE2289E59A116B02A26C82C8A8033DD8FA6D06A84B6501B7 /dev/sda2 /dev/sdb2: 52a87f000 ... 

这些命令可以针对不同的磁盘并行运行。 扫描一个38GB的分区需要大约12分钟。 在我的情况下，每8个驱动器中只发现一次32个字节的string。 通过比较bgrep返回的偏移量，你可以得到这样的图片：

 | offset \ partition | b | d | c | e | f | g | a | h | |--------------------+---+---+---+---+---+---+---+---| | 52a7ff000 | P | | | | | | | 1 | | 52a87f000 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | P | | 52a8ff000 | | | | | | | P | 9 | 

我们看到一个正常的左对称布局，这是mdadm默认布局。 更重要的是，现在我们知道分区的顺序。 但是，我们不知道哪个分区是数组中的第一个分区，因为它们可以循环移位。

还要注意find的偏移之间的距离。 在我的情况是512KB。 块大小实际上可能小于这个距离，在这种情况下，实际的布局将会不同。

查找原始块大小

我们使用相同的文件picture.jpg以相互不同的间隔读取32个字节的数据。 我们从上面知道，偏移量100k处的数据位于/dev/sdh2 ，偏移量612k处于/dev/sdb2 ，而1124k处于/dev/sdd2 。 这表明块大小不大于512KB。 我们validation它不小于512KB。 为此，我们转储字节串在偏移量356k，并查看它所在的分区：

 hexdump -n32 -s356k -v -e '/1 "%02X"' P1080801.JPG ; echo 7EC528AD0A8D3E485AE450F88E56D6AEB948FED7E679B04091B031705B6AFA7A sudo ./bgrep 7EC528AD0A8D3E485AE450F88E56D6AEB948FED7E679B04091B031705B6AFA7A /dev/sdb2 /dev/sdb2: 52a83f000 

它位于与偏移量612k相同的分区上，这表示块大小不是256KB。 我们以类似的方式消除较小的块大小。 我结束了512KB大块是唯一的可能性。

Find First Partition in Layout

Now we know the order of partitions, but we don't know which partition should be the first, and which RAID data offset was used. To find these two unknowns, we will create a RAID5 array with correct chunk layout and a small data offset, and search for the start of our file system in this new array.

To begin with, we create an array with the correct order of partitions, which we found earlier:

 sudo mdadm --stop /dev/md126 sudo mdadm --create /dev/md126 --assume-clean --raid-devices=8 --level=5 /dev/sdb2 /dev/sdd2 /dev/sdc2 /dev/sde2 /dev/sdf2 /dev/sdg2 /dev/sda2 /dev/sdh2 

We verify that the order is obeyed by issuing

 sudo mdadm --misc -D /dev/md126 ... Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 1 8 50 1 active sync /dev/sdd2 2 8 34 2 active sync /dev/sdc2 3 8 66 3 active sync /dev/sde2 4 8 82 4 active sync /dev/sdf2 5 8 98 5 active sync /dev/sdg2 6 8 2 6 active sync /dev/sda2 7 8 114 7 active sync /dev/sdh2 

Now we determine offsets of the N+1 known bytestrings in the RAID array. I run a script for a night (Live CD doesn't ask for password on sudo :):

 #!/bin/bash echo "1st:" sudo ./bgrep DA1DC4D616B1C71079624CDC36E3D40E7B1CFF00857C663687B6C4464D6C77D2 /dev/md126 echo "2nd:" sudo ./bgrep AB9DDDBBB05CA915EE2289E59A116B02A26C82C8A8033DD8FA6D06A84B6501B7 /dev/md126 echo "3rd:" sudo ./bgrep BC31A8DC791ACDA4FA3E9D3406D5639619576AEE2E08C03C9EF5E23F0A7C5CBA /dev/md126 ... echo "9th:" sudo ./bgrep 99B5A96F21BB74D4A630C519B463954EC096E062B0F5E325FE8D731C6D1B4D37 /dev/md126 

Output with comments:

 1st: /dev/md126: 2428fff000 # 1st 2nd: /dev/md126: 242947f000 # 480000 after 1st 3rd: # 3rd not found 4th: /dev/md126: 242917f000 # 180000 after 1st 5th: /dev/md126: 24291ff000 # 200000 after 1st 6th: /dev/md126: 242927f000 # 280000 after 1st 7th: /dev/md126: 24292ff000 # 300000 after 1st 8th: /dev/md126: 242937f000 # 380000 after 1st 9th: /dev/md126: 24297ff000 # 800000 after 1st 

Based on this data we see that the 3rd string was not found. This means that the chunk at /dev/sdd2 is used for parity. Here is an illustration of the parity positions in the new array:

 | offset \ partition | b | d | c | e | f | g | a | h | |--------------------+---+---+---+---+---+---+---+---| | 52a7ff000 | | | P | | | | | 1 | | 52a87f000 | 2 | P | 4 | 5 | 6 | 7 | 8 | | | 52a8ff000 | P | | | | | | | 9 | 

Our aim is to deduce which partition to start the array from, in order to shift the parity chunks into the right place. Since parity should be shifted two chunks to the left, the partition sequence should be shifted two steps to the right. Thus the correct layout for this data offset is ahbdcefg :

 sudo mdadm --stop /dev/md126 sudo mdadm --create /dev/md126 --assume-clean --raid-devices=8 --level=5 /dev/sda2 /dev/sdh2 /dev/sdb2 /dev/sdd2 /dev/sdc2 /dev/sde2 /dev/sdf2 /dev/sdg2 

At this point our RAID array contains data in the right form. You might be lucky so that the RAID data offset is the same as it was in the original array, and then you will most likely be able to mount the partition. Unfortunately this was not my case.

Verify Data Consistency

We verify that the data is consistent over a strip of chunks by extracting a copy of picture.jpg from the array. For this we locate the offset for the 32-byte string at 100k:

 sudo ./bgrep DA1DC4D616B1C71079624CDC36E3D40E7B1CFF00857C663687B6C4464D6C77D2 /dev/md126 

We then substract 100*1024 from the result and use the obtained decimal value in skip= parameter for dd . The count= is the size of picture.jpg in bytes:

 sudo dd if=/dev/md126 of=./extract.jpg bs=1 skip=155311300608 count=4536208 

Check that extract.jpg is the same as picture.jpg .

Find RAID Data Offset

A sidenote: default data offset for mdadm version 3.2.3 is 2048 sectors. But this value has been changed over time. If the original array used a smaller data offset than your current mdadm , then mdadm --create without --assume-clean can overwrite the beginning of the file system.

In the previous section we created a RAID array. Verify which RAID data offset it had by issuing for some of the individual partitions:

 sudo mdadm --examine /dev/sdb2 ... Data Offset : 2048 sectors ... 

2048 512-byte sectors is 1MB. Since chunk size is 512KB, the current data offset is two chunks.

If at this point you have a two-chunk offset, it is probably small enough, and you can skip this paragraph.
We create a RAID5 array with the data offset of one 512KB-chunk. Starting one chunk earlier shifts the parity one step to the left, thus we compensate by shifting the partition sequence one step to the left. Hence for 512KB data offset, the correct layout is hbdcefga . We use a version of mdadm that supports data offset (see Tools section). It takes offset in kilobytes:

 sudo mdadm --stop /dev/md126 sudo ./mdadm --create /dev/md126 --assume-clean --raid-devices=8 --level=5 /dev/sdh2:512 /dev/sdb2:512 /dev/sdd2:512 /dev/sdc2:512 /dev/sde2:512 /dev/sdf2:512 /dev/sdg2:512 /dev/sda2:512 

Now we search for a valid ext4 superblock. The superblock structure can be found here: https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#The_Super_Block
We scan the beginning of the array for occurences of the magic number s_magic followed by s_state and s_errors . The bytestrings to look for are:

 53EF01000100 53EF00000100 53EF02000100 53EF01000200 53EF02000200 

Example command:

 sudo ./bgrep 53EF01000100 /dev/md126 /dev/md126: 0dc80438 

The magic number starts 0x38 bytes into the superblock, so we substract 0x38 to calculate the offset and examine the entire superblock:

 sudo hexdump -n84 -s0xDC80400 -v /dev/md126 dc80400 2000 00fe 1480 03f8 cdd3 0032 d2b2 0119 dc80410 ab16 00f7 0000 0000 0002 0000 0002 0000 dc80420 8000 0000 8000 0000 2000 0000 b363 51bd dc80430 e406 5170 010d ffff ef53 0001 0001 0000 dc80440 3d3a 50af 0000 0000 0000 0000 0001 0000 dc80450 0000 0000 

This seems to be a valid superblock. s_log_block_size field at 0x18 is 0002, meaning that the block size is 2^(10+2)=4096 bytes. s_blocks_count_lo at 0x4 is 03f81480 blocks which is 254GB. 看起来不错。

We now scan for the occurrences of the first bytes of the superblock to find its copies. Note the byte flipping as compared to hexdump output:

 sudo ./bgrep 0020fe008014f803d3cd3200 /dev/md126 /dev/md126: 0dc80400 # offset by 1024 bytes from the start of the FS /dev/md126: 15c80000 # 32768 blocks from FS start /dev/md126: 25c80000 # 98304 /dev/md126: 35c80000 # 163840 /dev/md126: 45c80000 # 229376 /dev/md126: 55c80000 # 294912 /dev/md126: d5c80000 # 819200 /dev/md126: e5c80000 # 884736 /dev/md126: 195c80000 /dev/md126: 295c80000 

This aligns perfectly with the expected positions of backup superblocks:

 sudo mke2fs -n /dev/md126 ... Block size=4096 (log=2) ... Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872 

Hence the file system starts at the offset 0xdc80000, ie 225792KB from the partition start. Since we have 8 partitions of which one is for parity, we divide the offset by 7. This gives 33030144 bytes offset on every partition, which is exactly 63 RAID chunks. And since the current RAID data offset is one chunk, we conclude that the original data offset was 64 chunks, or 32768KB. Shifting hbdcefga 63 times to the right gives the layout bdcefgah .

We finally build the correct RAID array:

 sudo mdadm --stop /dev/md126 sudo ./mdadm --create /dev/md126 --assume-clean --raid-devices=8 --level=5 /dev/sdb2:32768 /dev/sdd2:32768 /dev/sdc2:32768 /dev/sde2:32768 /dev/sdf2:32768 /dev/sdg2:32768 /dev/sda2:32768 /dev/sdh2:32768 sudo fsck.ext4 -n /dev/md126 e2fsck 1.42 (29-Nov-2011) Warning: skipping journal recovery because doing a read-only filesystem check. /dev/md126: clean, 423146/16654336 files, 48120270/66589824 blocks sudo mount -t ext4 -r /dev/md126 /home/xubuntu/mp 

瞧！

我有一个类似的问题。 I formatted and reinstalled my OS/boot drive with a clean install of Ubuntu 12.04, then ran the mdadm –create… command and couldn't mount it.

It said it didn't have a valid superblock or partition.

Moreover, when I stopped the mdadm raid, I could no longer mount the regular device.

I was able to repair the superblock with mke2fs and e2fsck:

 root@blackbox:~# mke2fs -n /dev/sdc1 mke2fs 1.42 (29-Nov-2011) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 91578368 inodes, 366284000 blocks 18314200 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=4294967296 11179 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000, 214990848 

Then ran:

 e2fsck -b 32768 -y /dev/sdc1 

That restored the superblock so I could mount and read the drive.

To get the array working without destroying the superblock or partitions I used build:

 mdadm --build /dev/md0 --level=mirror --assume-clean --raid-devices=2 /dev/sdc1 missing 

After verifying the data, I will add the other drive:

 mdadm --add /dev/md0 /sdd1 

I'm just updating some of the information given earlier. I had a 3-disk raid5 array working ok when my motherboard died. The array held /dev/md2 as the /home partition 1.2TB and /dev/md3 as the /var partition 300GB.

I had two backups of "important" stuff and a bunch of random things I had grabbed from various parts of the internet that I really should have gone through and selectively dumped. Most of the backups were broken into .tar.gz files of 25GB or less, and a separate copy of /etc was backed up also.

The rest of the filesystem was held on two small raid0 disks of 38GB.

My new machine was similar to the old hardware, and I got the machine up and running simply by plugging all five disks in and selecting an old generic kernel. So I had five disks with clean filesystems, though I could not be certain that the disks were in the right order, and needed to install a new version of Debian Jessie to be sure that I could upgrade the machine when needed, and sort out other problems.

With the new generic system installed on two Raid0 disks, I began to put the arrays back together. I wanted to be sure that I had the disks in the right order. What I should have done was to issue :

 mdadm --assemble /dev/md3 -o --no-degraded --uuid=82164ae7:9af3c5f1:f75f70a5:ba2a159a 

但是我没有。 It seems that mdadm is pretty smart and given a uuid, can figure out which drives go where. Even if the bios designates /dev/sdc as /sda, mdadm will put it together correctly (YMMV though).

Instead I issued: mdadm --create /dev/md2 without the --assume-clean , and allowed the resync on /dev/sde1 to complete. The next mistake I made was to work on /dev/sdc1 instead of the last drive in the /dev/md2, /sde1. Anytime mdadm thinks there is a problem it is the last drive that gets kicked out or re-synced.

After that, mdadm could not find any superblock, and e2fsck -n couldn't either.

After I found this page, I went through the procedure of trying to find the sequence for the drives (done), check for valid data (verified 6MB of a 9MB file), got the disks in the right sequence, cde, grabbed the uuid's of /md2 and /md3 from the old /etc/mdadm.conf and tried assembling.

Well, /dev/md3 started, and mdadm --misc -D /dev/md3 showed three healthy partitions, and the disks in the right order. /dev/md2 also looked good, until I tried to mount the filesystem.

 # mdadm --create /dev/md2 --raid-devices=3 --level=5 --uuid=c0a644c7:e5bcf758:ecfbc8f3:ee0392b7 /dev/sdc1 /dev/sdd1 /dev/sde1 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Feb 3 14:05:36 2016 mdadm: /dev/sdd1 appears to contain an ext2fs file system size=585936896K mtime=Thu Jan 1 01:00:00 1970 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Feb 3 14:05:36 2016 mdadm: /dev/sde1 appears to contain an ext2fs file system size=585936896K mtime=Thu Jan 1 01:00:00 1970 mdadm: /dev/sde1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Feb 3 14:05:36 2016 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md2 started. 

The filesystem refused to be mounted, and e2fsck couldn't find any superblocks. Further, when checking for superblocks as described above, the total block count found as a880 0076 or a880 0076 or 5500 1176 did not match the disk capacity size of 1199.79 reported my mdadm. Also none of the locations of the "superblocks" aligned with the data in the posts above.

I backed up all of /var, and prepared to wipe the disks. To see if it was possible to wipe just /md2, (I had nothing else to lose at this point) I dis the following:

 root@ced2:/home/richard# mdadm --create /dev/md2 --raid-devices=3 --level=5 --uuid=c0a644c7:e5bcf758:ecfbc8f3:ee0392b7 /dev/sdc1 /dev/sdd1 /dev/sde1 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Feb 3 14:05:36 2016 mdadm: /dev/sdd1 appears to contain an ext2fs file system size=585936896K mtime=Thu Jan 1 01:00:00 1970 mdadm: /dev/sdd1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Feb 3 14:05:36 2016 mdadm: /dev/sde1 appears to contain an ext2fs file system size=585936896K mtime=Thu Jan 1 01:00:00 1970 mdadm: /dev/sde1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Feb 3 14:05:36 2016 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md2 started. # mkfs.ext3 /dev/md2 mke2fs 1.42.12 (29-Aug-2014) Creating filesystem with 292902912 4k blocks and 73228288 inodes Filesystem UUID: a54e252f-78db-4ebb-b7ca-7dcd2edf57a4 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000, 214990848 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done # hexdump -n84 -s0x00000400 -v /dev/md2 0000400 6000 045d 5800 1175 7799 00df 6ff0 112e 0000410 5ff5 045d 0000 0000 0002 0000 0002 0000 0000420 8000 0000 8000 0000 2000 0000 10d3 56b2 0000430 10d3 56b2 0002 ffff ef53 0001 0001 0000 0000440 0c42 56b2 0000 0000 0000 0000 0001 0000 0000450 0000 0000 0000454 # ./bgrep 00605D0400587511 /dev/md2 /dev/md2: 00000400 /dev/md2: 08000000 /dev/md2: 18000000 /dev/md2: 28000000 /dev/md2: 38000000 /dev/md2: 48000000 /dev/md2: c8000000 /dev/md2: d8000000 /dev/md2: 188000000 /dev/md2: 288000000 /dev/md2: 3e8000000 /dev/md2: 798000000 /dev/md2: ab8000000 etc 

All seemed ok, except for the change to the uuid. So after a couple more checks, I wrote 600GB of backed up data onto /dev/md2. Then, unmounted and tried to re-mount the drive:

 # mdadm --assemble /dev/md2 uuid=c0a644c7:e5bcf758:ecfbc8f3:ee0392b7 mdadm: cannot open device uuid=c0a644c7:e5bcf758:ecfbc8f3:ee0392b7: No such file or directory mdadm: uuid=c0a644c7:e5bcf758:ecfbc8f3:ee0392b7 has no superblock - assembly aborted 

Are you ********* kidding me? what about my 600GB on the file?

 # mdadm --assemble /dev/md2 mdadm: /dev/md2 not identified in config file. 

Ah – easily fixed. uncommented one line in /etc/mdadm.conf

 # mdadm --assemble /dev/md2 mdadm: /dev/md2 has been started with 3 drives. # e2fsck -n /dev/md2 e2fsck 1.42.12 (29-Aug-2014) /dev/md2: clean, 731552/73228288 files, 182979586/292902912 blocks 

Yippie!