在意外断电后，我正在恢复运行Scientific Linux 5.5的Dell Poweredge R510服务器。 服务器是由我们以前的系统pipe理员（我是一个研究生）设置的。 重新启动后，我看到消息：
fsck.ext3: Device or resource busy when trying to open /dev/sdb1 File system mounted or opened exclusively by another program? / dev / sdb1是/ home目录，由由9×600 GB SAS磁盘组成的Megaraid控制的RAID5组成。
# megacli -AdpAllInfo -aALL Adapter #0 ============================================================================== Versions ================ Product Name : PERC H700 Integrated Serial No : 06C006X FW Package Build: 12.3.0-0032 Mfg. Data ================ Mfg. Date : 06/19/10 Rework Date : 06/19/10 Revision No : A00 Battery FRU : N/A Image Versions in Flash: ================ BIOS Version : 3.09.00 FW Version : 2.30.03-0872 Preboot CLI Version: 04.02-004:#%00008 Ctrl-R Version : 2.02-0009 NVDATA Version : 2.03.0053 Boot Block Version : 2.02.00.00-0000 BOOT Version : 01.250.04.219 Pending Images in Flash ================ None PCI Info ================ Vendor Id : 1000 Device Id : 0079 SubVendorId : 1028 SubDeviceId : 1f17 Host Interface : PCIE Number of Frontend Port: 0 Device Interface : PCIE Number of Backend Port: 8 Port : Address 0 500065b36789abff 1 0000000000000000 2 0000000000000000 3 0000000000000000 4 0000000000000000 5 0000000000000000 6 0000000000000000 7 0000000000000000 HW Configuration ================ SAS Address : 5842b2b01789d900 BBU : Present Alarm : Absent NVRAM : Present Serial Debugger : Present Memory : Present Flash : Present Memory Size : 1024MB TPM : Absent On board Expander: Absent Upgrade Key : Absent Settings ================ Current Time : 3:20:56 1/13, 2013 Predictive Fail Poll Interval : 300sec Interrupt Throttle Active Count : 16 Interrupt Throttle Completion : 50us Rebuild Rate : 30% PR Rate : 30% BGI Rate : 30% Check Consistency Rate : 30% Reconstruction Rate : 30% Cache Flush Interval : 4s Max Drives to Spinup at One Time : 4 Delay Among Spinup Groups : 12s Physical Drive Coercion Mode : 128MB Cluster Mode : Disabled Alarm : Disabled Auto Rebuild : Enabled Battery Warning : Enabled Ecc Bucket Size : 15 Ecc Bucket Leak Rate : 1440 Minutes Restore HotSpare on Insertion : Disabled Expose Enclosure Devices : Disabled Maintain PD Fail History : Disabled Host Request Reordering : Enabled Auto Detect BackPlane Enabled : SGPIO/i2c SEP Load Balance Mode : Auto Use FDE Only : Yes Security Key Assigned : No Security Key Failed : No Security Key Not Backedup : No Any Offline VD Cache Preserved : No Allow Boot with Preserved Cache : No Disable Online Controller Reset : No PFK in NVRAM : No Use disk activity for locate : No Capabilities ================ RAID Level Supported : RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span Supported Drives : SAS, SATA Allowed Mixing: Mix in Enclosure Allowed Status ================ ECC Bucket Count : 0 Limitations ================ Max Arms Per VD : 32 Max Spans Per VD : 8 Max Arrays : 128 Max Number of VDs : 64 Max Parallel Commands : 1008 Max SGE Count : 60 Max Data Transfer Size : 8192 sectors Max Strips PerIO : 42 Min Strip Size : 8 KB Max Strip Size : 1.0 MB Max Configurable CacheCade Size: 0 GB Current Size of CacheCade : 0 GB Current Size of FW Cache : 0 MB Device Present ================ Virtual Drives : 2 Degraded : 0 Offline : 0 Physical Devices : 14 Disks : 12 Critical Disks : 0 Failed Disks : 0 Supported Adapter Operations ================ Rebuild Rate : Yes CC Rate : Yes BGI Rate : Yes Reconstruct Rate : Yes Patrol Read Rate : Yes Alarm Control : Yes Cluster Support : No BBU : Yes Spanning : Yes Dedicated Hot Spare : Yes Revertible Hot Spares : Yes Foreign Config Import : Yes Self Diagnostic : Yes Allow Mixed Redundancy on Array : No Global Hot Spares : Yes Deny SCSI Passthrough : No Deny SMP Passthrough : No Deny STP Passthrough : No Support Security : Yes Snapshot Enabled : No Support the OCE without adding drives : Yes Support PFK : No Supported VD Operations ================ Read Policy : Yes Write Policy : Yes IO Policy : Yes Access Policy : Yes Disk Cache Policy : Yes Reconstruction : Yes Deny Locate : No Deny CC : No Allow Ctrl Encryption: No Enable LDBBM : Yes Supported PD Operations ================ Force Online : Yes Force Offline : Yes Force Rebuild : Yes Deny Force Failed : No Deny Force Good/Bad : No Deny Missing Replace : No Deny Clear : No Deny Locate : No Disable Copyback : No Enable JBOD : No Enable Copyback on SMART : No Enable Copyback to SSD on SMART Error : No Enable SSD Patrol Read : No PR Correct Unconfigured Areas : Yes Enable Spin Down of UnConfigured Drives : No Disable Spin Down of hot spares : Yes Spin Down time : 30 Error Counters ================ Memory Correctable Errors : 0 Memory Uncorrectable Errors : 0 Cluster Information ================ Cluster Permitted : No Cluster Active : No Default Settings ================ Phy Polarity : 0 Phy PolaritySplit : 0 Background Rate : 30 Strip Size : 64kB Flush Time : 4 seconds Write Policy : WB Read Policy : Adaptive Cache When BBU Bad : Disabled Cached IO : No SMART Mode : Mode 6 Alarm Disable : Yes Coercion Mode : 128MB ZCR Config : Unknown Dirty LED Shows Drive Activity : No BIOS Continue on Error : No Spin Down Mode : None Allowed Device Type : SAS/SATA Mix Allow Mix in Enclosure : Yes Allow HDD SAS/SATA Mix in VD : No Allow SSD SAS/SATA Mix in VD : No Allow HDD/SSD Mix in VD : No Allow SATA in Cluster : No Max Chained Enclosures : 1 Disable Ctrl-R : No Enable Web BIOS : No Direct PD Mapping : Yes BIOS Enumerate VDs : Yes Restore Hot Spare on Insertion : No Expose Enclosure Devices : No Maintain PD Fail History : No Disable Puncturing : No Zero Based Enclosure Enumeration : Yes PreBoot CLI Enabled : No LED Show Drive Activity : Yes Cluster Disable : Yes SAS Disable : No Auto Detect BackPlane Enable : SGPIO/i2c SEP Use FDE Only : Yes Enable Led Header : No Delay during POST : 0 EnableCrashDump : No Disable Online Controller Reset : No EnableLDBBM : Yes Un-Certified Hard Disk Drives : Allow Treat Single span R1E as R10 : Yes Max LD per array : 16 Power Saving option : Disable all power saving options Default spin down time in minutes: 30 Enable JBOD : No Exit Code: 0x00
在错误显示之前的dmesg ：
device-mapper: multipath: version 1.0.5 loaded device-mapper: multipath round-robin: version 1.0.0 device-mapper: table 253:0: multipath: error getting device device-mapper: ioctl: error: adding target to table device-mapper: table 253:0: multipath: error getting device device-mapper: ioctl: error: adding target to table
如果我注释掉/ etc / fstab文件中的相应条目并重启
LABEL=/home /home ext3 defaults 1 2
系统正常引导（但没有磁盘）。 但是，我仍然无法安装磁盘。 稍作调查进一步产生以下结果：
# mount /dev/sdb1 /home mount: /dev/sdb1 already mounted or /home busy # lsof /dev/sdb COMMAND PID USER FD TYPE DEVICE SIZE MODE NAME multipath 3864 root 5r BLK 8,16 2582 /dev/sdb # fuser /dev/sdb 3864 #ps -ef | grep 3864 3864 1 0 19:22 ? 00:00:00 /sbin/multipathd
显然多path阻止我手动安装磁盘。 杀死多path守护进程是否安全或适当？ multipathdpath的path和configuration如下：
multipathd> show paths hcil dev dev_t pri dm_st chk_st next_check 0:2:0:0 sda 8:0 1 [undef] [ready] [orphan] 0:2:1:0 sdb 8:16 1 [active][ready] XXXXXX.... 13/20 multipathd> show config defaults { verbosity 2 user_friendly_names yes } blacklist { devnode ^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]* devnode ^hd[az] device { vendor DGC product LUNZ } device { vendor EMC product LUNZ } device { vendor IBM product S/390.* } device { vendor IBM product S/390.* } device { vendor STK product Universal Xport } } blacklist_exceptions { } devices { device { vendor NETAPP product LUN path_grouping_policy multibus path_checker directio features 1 queue_if_no_path prio_callout /sbin/mpath_prio_ontap /dev/%n failback immediate flush_on_last_del yes } device { vendor APPLE* product Xserve RAID path_grouping_policy multibus } device { vendor 3PARdata product VV path_grouping_policy multibus } device { vendor DEC product HSG80 path_grouping_policy group_by_prio path_checker hp_sw features 1 queue_if_no_path hardware_handler 1 hp-sw prio_callout /sbin/mpath_prio_hp_sw /dev/%n } device { vendor COMPAQ product (MSA|HSV)1.0.* path_grouping_policy group_by_prio path_checker hp_sw features 1 queue_if_no_path hardware_handler 1 hp-sw prio_callout /sbin/mpath_prio_hp_sw /dev/%n no_path_retry 12 rr_min_io 100 } device { vendor (COMPAQ|HP) product HSV1[01]1|HSV2[01]0|HSV300|HSV4[05]|HSV4[05]0 path_grouping_policy group_by_prio path_checker tur prio_callout /sbin/mpath_prio_alua /dev/%n failback immediate no_path_retry 12 rr_min_io 100 } device { vendor (COMPAQ|HP) product MSA VOLUME path_grouping_policy group_by_prio path_checker tur prio_callout /sbin/mpath_prio_alua /dev/%n failback immediate no_path_retry 12 rr_min_io 100 } device { vendor HP product MSA2[02]12fc|MSA2012i path_grouping_policy multibus path_checker tur prio_callout /bin/true failback immediate no_path_retry 18 rr_min_io 100 } device { vendor HP product MSA2012sa|MSA23(12|24)(fc|i|sa)|MSA2000s VOLUME path_grouping_policy group_by_prio path_checker tur prio_callout /sbin/mpath_prio_alua /dev/%n failback immediate no_path_retry 18 rr_min_io 100 } device { vendor HP product HSVX700 path_grouping_policy group_by_prio path_checker tur hardware_handler 1 alua prio_callout /sbin/mpath_prio_alua /dev/%n failback immediate no_path_retry 12 rr_min_io 100 } device { vendor HP product A6189A path_grouping_policy multibus } device { vendor DDN product SAN DataDirector path_grouping_policy multibus } device { vendor EMC product SYMMETRIX path_grouping_policy multibus getuid_callout /sbin/scsi_id -g -u -ppre-spc3-83 -s /block/%n } device { vendor DGC product .* product_blacklist LUNZ path_grouping_policy group_by_prio path_checker emc_clariion features 1 queue_if_no_path hardware_handler 1 emc prio_callout /sbin/mpath_prio_emc /dev/%n failback immediate no_path_retry 60 } device { vendor FSC product CentricStor path_grouping_policy group_by_serial } device { vendor (HITACHI|HP) product OPEN-.* path_grouping_policy multibus path_checker tur failback immediate no_path_retry 12 } device { vendor HITACHI product DF.* path_grouping_policy group_by_prio prio_callout /sbin/mpath_prio_hds_modular %d failback immediate } device { vendor EMC product Invista product_blacklist LUNZ path_grouping_policy multibus path_checker tur no_path_retry 5 } device { vendor IBM product ProFibre 4000R path_grouping_policy multibus } device { vendor IBM product 1722-600 path_grouping_policy group_by_prio path_checker rdac features 1 queue_if_no_path hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry 300 } device { vendor IBM product 1724 path_grouping_policy group_by_prio path_checker rdac features 1 queue_if_no_path hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry 300 } device { vendor IBM product 1726 path_grouping_policy group_by_prio path_checker rdac features 1 queue_if_no_path hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry 300 } device { vendor IBM product 1742 path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate } device { vendor IBM product 1814 path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry queue } device { vendor IBM product 1745|1746 path_grouping_policy group_by_prio path_checker rdac features 2 pg_init_retries 50 hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry 15 } device { vendor IBM product 1815 path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry queue } device { vendor IBM product 1818 path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry queue } device { vendor IBM product 3526 path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate } device { vendor IBM product 3542 path_grouping_policy group_by_serial path_checker tur } device { vendor IBM product 2105(800|F20) path_grouping_policy group_by_serial path_checker tur features 1 queue_if_no_path } device { vendor IBM product 1750500 path_grouping_policy group_by_prio path_checker tur features 1 queue_if_no_path prio_callout /sbin/mpath_prio_alua /dev/%n failback immediate } device { vendor IBM product 2107900 path_grouping_policy multibus path_checker tur features 1 queue_if_no_path } device { vendor IBM product 2145 path_grouping_policy group_by_prio path_checker tur features 1 queue_if_no_path prio_callout /sbin/mpath_prio_alua /dev/%n failback immediate } device { vendor IBM product S/390 DASD ECKD product_blacklist S/390.* path_grouping_policy multibus getuid_callout /sbin/dasd_id /dev/%n path_checker directio features 1 queue_if_no_path } device { vendor IBM product S/390 DASD FBA product_blacklist S/390.* path_grouping_policy multibus getuid_callout /sbin/dasd_id /dev/%n path_checker directio } device { vendor NETAPP product LUN.* path_grouping_policy group_by_prio path_checker directio features 1 queue_if_no_path prio_callout /sbin/mpath_prio_ontap /dev/%n failback immediate rr_min_io 128 } device { vendor IBM product Nseries.* path_grouping_policy group_by_prio features 1 queue_if_no_path prio_callout /sbin/mpath_prio_ontap /dev/%n failback immediate rr_min_io 128 } device { vendor Pillar product Axiom [35]00 path_grouping_policy group_by_prio path_checker tur prio_callout /sbin/mpath_prio_alua %d } device { vendor AIX product VDASD path_grouping_policy multibus path_checker directio failback immediate no_path_retry 60 } device { vendor SGI product TP9[13]00 path_grouping_policy multibus } device { vendor SGI product TP9[45]00 path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate } device { vendor SGI product IS.* path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry queue } device { vendor STK product OPENstorage D280 path_grouping_policy group_by_prio path_checker tur hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate } device { vendor STK product FLEXLINE 380 product_blacklist Universal Xport path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry queue } device { vendor SUN product (StorEdge 3510|T4) path_grouping_policy multibus } device { vendor PIVOT3 product RAIGE VOLUME path_grouping_policy multibus getuid_callout /sbin/scsi_id -p 0x80 -g -u -d /dev/%n path_checker tur features 1 queue_if_no_path rr_min_io 100 } device { vendor SUN product CSM200_R path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry queue } device { vendor SUN product LCSM100_F path_grouping_policy group_by_prio path_checker rdac hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry queue } device { vendor (LSI|ENGENIO) product INF.* path_grouping_policy group_by_prio path_checker rdac features 2 pg_init_retries 50 hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry 15 } device { vendor DELL product MD3000|MD3000i path_grouping_policy group_by_prio path_checker rdac features 2 pg_init_retries 50 hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry 15 } device { vendor DELL product MD32xx|MD32xxi path_grouping_policy group_by_prio path_checker rdac features 2 pg_init_retries 50 hardware_handler 1 rdac prio_callout /sbin/mpath_prio_rdac /dev/%n failback immediate no_path_retry 15 } device { vendor COMPELNT product Compellent Vol path_grouping_policy multibus path_checker tur failback immediate no_path_retry queue } device { vendor GNBD product GNBD path_grouping_policy multibus getuid_callout /sbin/gnbd_import -q -U /block/%n path_checker directio } } multipaths { }
我的/etc/multipath.conf文件的内容：
# Blacklist all local devices devnode_blacklist { devnode "sd[ab]$" devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[az]" devnode "^cciss!c[0-9]d[0-9]*" } ## Use user friendly names, instead of using WWIDs as names. defaults { user_friendly_names yes } devices { device { vendor "NETAPP" product "LUN" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout "/sbin/mpath_prio_ontap /dev/%n" features "1 queue_if_no_path" path_checker directio failback immediate flush_on_last_del yes } }
驱动器本身看起来状况良好。 如果我使用USB加载的LiveCD .iso，我可以挂载/ dev / sdb1没有任何问题。 这些文件似乎都存在。 运行fsck -ylv /dev/sdb1 ，磁盘看起来很好。 输出如下所示：
# fsck.ext3 -fyv /dev/sdc1 e2fsck 1.39 (29-May-2006) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information 1387340 inodes used (0.12%) 219427 non-contiguous inodes (15.8%) # of inodes with ind/dind/tind blocks: 343585/63330/32 869857188 blocks used (74.28%) 0 bad blocks 96 large files 1310760 regular files 71629 directories 0 character device files 0 block device files 0 fifos 1 link 4942 symbolic links (4497 fast symbolic links) 0 sockets -------- 1387332 files
为了完整性，输出fdisk -l ：
WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted. WARNING: The size of this disk is 4.8 TB (4796404727808 bytes). DOS partition table format can not be used on drives for volumes larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID partition table format (GPT). WARNING: GPT (GUID Partition Table) detected on '/dev/dm-0'! The util fdisk doesn't support GPT. Use GNU Parted. WARNING: The size of this disk is 4.8 TB (4796404727808 bytes). DOS partition table format can not be used on drives for volumes larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID partition table format (GPT). Disk /dev/dm-1 doesn't contain a valid partition table Disk /dev/sda: 599.5 GB, 599550590976 bytes 255 heads, 63 sectors/track, 72891 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 5099 40957686 83 Linux /dev/sda2 5100 69194 514843087+ 83 Linux /dev/sda3 69195 71744 20482875 82 Linux swap / Solaris /dev/sda4 71745 72891 9213277+ 5 Extended /dev/sda5 71745 72381 5116671 83 Linux /dev/sda6 72382 72891 4096543+ 83 Linux Disk /dev/sdb: 4796.4 GB, 4796404727808 bytes 255 heads, 63 sectors/track, 583129 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 267350 2147483647+ ee EFI GPT Disk /dev/dm-0: 4796.4 GB, 4796404727808 bytes 255 heads, 63 sectors/track, 583129 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/dm-0p1 1 267350 2147483647+ ee EFI GPT Disk /dev/dm-1: 4796.4 GB, 4796404693504 bytes 255 heads, 63 sectors/track, 583129 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes
我怀疑，更改configuration文件中的一些设置，无论是为多path或其他实用程序，将解决我的问题。 但是，我不知道如何继续。
您可以将驱动器黑名单，多path将跳过它。 放：
blacklist { devnode "sd[ab]" } defaults { user_friendly_names yes }
在/etc/multipath.conf重启。 它看起来像文件系统的一切都完好无损，所以不要担心。 当你发出lsof它应该在分区上，而不是在整个设备上（ lsof /dev/sdb1而不是lsof /dev/sdb ）。 fuser 。 但是，先尝试黑名单，因为它可能只是你需要的。
在你的/etc/multipath.conf文件中改变：
devnode_blacklist {
devnode“sd [ab] $”
…
}
至：
devnode_blacklist {
devnode“sd [ab] *”
…
}
这将黑名单/ dev / sdb1，而您当前的configuration不会黑名单/ dev / sdb1
也许这有帮助。 在我的情况下，我遇到了多path和“设备或资源繁忙”的问题。 我使用multipath -l列出了所有的映射。 然后，我用multipath -f <MAPPING NAME>一个接一个地删除映射。 您可能可以使用multipath -F一次删除它们。 然后我能够创buildRAID。
我必须注意到，这些硬盘已经被清零了，我在这些硬盘上创build了一个大的GPT分区。 /etc/multipath.conf的黑名单似乎也是合理的。 将数组放入数组时，您不希望多path触碰硬盘。