热插拔SAS驱动器有问题。
初始数据:Nexenta 4.0.2基于服务器的Supermicro(MB S5520HC,内部RAID控制器RMS2LL080 / LSI 2008),12个HDD SAS 300G 10个HDD SATA 1T,2个SSD 160G。
磁盘分为三个池:
在某种程度上,两个SAS驱动器无序:
NAME STATE READ WRITE CKSUM sas DEGRADED 0 0 0 raidz1-0 DEGRADED 0 0 0 c1t5000C50007DCF821d0 ONLINE 0 0 0 c1t5000CCA0052FFDD5d0 ONLINE 0 0 0 c1t5000CCA005349D15d0 ONLINE 0 0 0 spare-3 FAULTED 0 0 0 c1t5000CCA00534D625d0 FAULTED 0 0 0 external device fault c1t5000CCA0053658B5d0 ONLINE 0 0 0 c1t5000CCA00534F2D5d0 ONLINE 0 0 0 c1t5000CCA00534F33Dd0 ONLINE 0 0 0 c1t5000CCA00534F471d0 ONLINE 0 0 0 c1t5000CCA0053571D1d0 ONLINE 0 0 0 c1t5000CCA00535A3A5d0 ONLINE 0 0 0 logs c0t500151795950C847d0 ONLINE 0 0 0 spares c1t5000CCA0053658B5d0 INUSE currently in use pool: syspool state: DEGRADED status: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the faulted device, or use 'zpool clear' to mark the device repaired. scan: scrub repaired 0 in 0h3m with 0 errors on Sun Oct 4 03:03:56 2015 config: NAME STATE READ WRITE CKSUM syspool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 c1t5000C500072BB235d0s0 FAULTED 1 20 0 external device fault c1t5000C500072BE655d0s0 ONLINE 0 0 0
接下来,为每个磁盘创build一个replace程序zpool offline / detach和cfgadm -c unconfigure。 在日志中安装一个驱动器(在这种情况下,syspool)后,可以看到以下内容:
Oct 9 18:28:43 nxstor genunix: [ID 408114 kern.info] /pci@0,0/pci8086,3a40@1c/pci8086,350e@0/iport@ff/disk@w5000c50007dcf7ed,0 (sd26) online Oct 9 18:28:43 nxstor pcplusmp: [ID 805372 kern.info] pcplusmp: pci-ide (pci-ide) instance 0 irq 0x12 vector 0x41 ioapic 0x8 intin 0x12 is bound to cpu 14 Oct 9 18:28:43 nxstor pcplusmp: [ID 805372 kern.info] pcplusmp: pci-ide (pci-ide) instance 1 irq 0x15 vector 0x42 ioapic 0x8 intin 0x15 is bound to cpu 15 Oct 9 18:28:43 nxstor pcplusmp: [ID 805372 kern.info] pcplusmp: pci-ide (pci-ide) instance 0 irq 0x12 vector 0x41 ioapic 0x8 intin 0x12 is bound to cpu 0 Oct 9 18:28:43 nxstor pcplusmp: [ID 805372 kern.info] pcplusmp: pci-ide (pci-ide) instance 1 irq 0x15 vector 0x41 ioapic 0x8 intin 0x15 is bound to cpu 1 Oct 9 18:39:53 nxstor genunix: [ID 888150 kern.warning] WARNING: Device not found in device tree. Skipping device unretire: /pci@0,0/pci8086,3a40@1c/pci8086,350e@0/iport@ff/disk@w5000c500072bb235,0 Oct 9 18:39:53 nxstor genunix: [ID 484473 kern.notice] NOTICE: Not retired: /pci@0,0/pci8086,3a40@1c/pci8086,350e@0/iport@ff/disk@w5000c500072bb235,0 Oct 9 18:39:53 nxstor genunix: [ID 888150 kern.warning] WARNING: Device not found in device tree. Skipping device unretire: /pci@0,0/pci8086,3a40@1c/pci8086,350e@0/iport@ff/disk@w5000cca00534d625,0
磁盘和磁盘@ w5000cca00534d625,0磁盘@ w5000c500072bb235,0是两个有问题的磁盘,已从系统中删除。
安装硬盘定义的cfgadm:
root@nxstor:/volumes# cfgadm -al Ap_Id Type Receptacle Occupant Condition Slot2 sas/hp connected configured ok c1 scsi-sas connected configured unknown c1::dsk/c1t5000C50007DCF7EDd0 disk connected configured unknown <cut>
但是,这种光盘格式也没有看到实用程序fdisk:
root@nxstor:/volumes# fdisk /dev/rdsk/c1t5000C50007DCF7EDd0 fdisk: Cannot stat device /dev/rdsk/c1t5000C50007DCF7EDd0. root@nxstor:/volumes# root@nxstor:/volumes# ls -la /dev/rdsk/c1t5000C50007DCF7EDd0 /dev/rdsk/c1t5000C50007DCF7EDd0: No such file or directory
而且,NMS看到以下内容:
Trigger Name: nms-fmacheck Fault ID: 5 Error Repeat Count: 5 Error Severity: CRITICAL Error TimeStamp: Tue Oct 13 14:21:54 2015 Description: FMA Module: ereport.io.scsi.disk.attach-failure Details: List of last errors : Oct 13 13:48:02.6970 ereport.io.scsi.cmd.disk.tran <cut> Oct 13 14:21:05.7075 ereport.io.scsi.cmd.disk.tran Oct 13 14:21:53.6196 ereport.io.scsi.cmd.disk.dev.rqs.derr Oct 13 14:21:53.6197 ereport.io.scsi.disk.attach-failure List of last errors : =========: Event Details :======== SOURCE: ereport.io.scsi.disk.attach-failure nvlist version: 0 class = ereport.io.scsi.disk.attach-failure ena = 0x58420a3d24100401 detector = (embedded nvlist) nvlist version: 0 version = 0x0 scheme = dev device-path = /pci@0,0/pci8086,3a40@1c/pci8086,350e@0/iport@ff/disk@w5000c50007dcf7ed,0 devid = id1,sd@n5000c50007dcf7ef (end detector) devid = id1,sd@n5000c50007dcf7ef __ttl = 0x1 __tod = 0x561cdb41 0x24f05942 =========
如果我们假设新驱动器也有缺陷,并把其他人得到更奇怪的图片。 日志安装盘完全不出现。 但信息显示在光盘的提取上:
Oct 13 14:50:26 nxstor scsi: [ID 107833 kern.notice] /pci@0,0/pci8086,3a40@1c/pci8086,350e@0 (mpt_sas0): Oct 13 14:50:26 nxstor PhysDiskNum 2 with DevHandle 0x23 in slot 0 for enclosure with handle 0x0 is now offline
实际上,已经发现这个问题只适用于SAS驱动器。 这反过来已经提出了与MPXIO驱动程序和mpt_sas(这是一个已知的问题solaris,并写道,她已经解决,但不清楚是否它通过opensolaris)在cfgadm中的问题。
接下来做什么? 谁遇到类似的问题?
驱动程序和企业级SAS控制器:MPTSAS HBA驱动程序00.00.00.24固件版本5.40.1.0 SAS驱动两种:Hitachi Ultrastar 15K300 HUS153030VLS300和希捷Cheetah 15K.5 ST3300655SS。