今天早上,我在FreeNAS 9.3机器上发出警告,提示“空间不足”
warning: The capacity for the volume 'SeanVolume' is currently at 85%, while the recommended value is below 80%.
一段时间后运行scrub命令是没有用的。
我使用SSHlogin到FreeNAS主机,并获得一些信息:
[root@freenas] ~# zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT SeanVolume 21.8T 18.6T 3.16T - 38% 85% 1.00x ONLINE /mnt freenas-boot 111G 680M 110G - - 0% 1.00x ONLINE -
它显示它的ALLOC空间只有18.6T的容量,但它仍然有6.7T通过du -sh命令:
[root@freenas] ~# du -sh /mnt/SeanVolume/ 6.7T /mnt/SeanVolume/
附加信息:
[root@freenas] ~# zpool status pool: SeanVolume state: ONLINE scan: scrub repaired 0 in 20h32m with 0 errors on Thu Jul 16 07:48:34 2015 config: NAME STATE READ WRITE CKSUM SeanVolume ONLINE 0 0 0 raidz3-0 ONLINE 0 0 0 gptid/f4986ea8-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/f51a05f7-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/f595d78e-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/f62490d4-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/f6a8e41e-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/f7266471-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/f7f3bf28-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/f8b8fa7b-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 cache gptid/f901ec00-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0 gptid/dfa53351-1baa-11e5-ba0e-d05099265144 ONLINE 0 0 0 errors: No known data errors
df -TH命令的结果:
[root@freenas] ~# df -TH Filesystem Type Size Used Avail Capacity Mounted on freenas-boot/ROOT/default zfs 115G 698M 114G 1% / devfs devfs 1.0k 1.0k 0B 100% /dev tmpfs tmpfs 33M 5.5M 28M 16% /etc tmpfs tmpfs 4.2M 8.2k 4.2M 0% /mnt tmpfs tmpfs 11G 67M 11G 1% /var freenas-boot/grub zfs 114G 8.1M 114G 0% /boot/grub SeanVolume zfs 1.6T 822k 1.6T 0% /mnt/SeanVolume SeanVolume/CandyDataset zfs 1.6T 299k 1.6T 0% /mnt/SeanVolume/CandyDataset SeanVolume/CandyDataset/Applications zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Applications SeanVolume/CandyDataset/Documents zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Documents SeanVolume/CandyDataset/Downloads zfs 2.9T 1.3T 1.6T 46% /mnt/SeanVolume/CandyDataset/Downloads SeanVolume/CandyDataset/Music zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Music SeanVolume/CandyDataset/Pictures zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Pictures SeanVolume/CandyDataset/Temporary zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Temporary SeanVolume/CandyDataset/Videos zfs 1.8T 221G 1.6T 12% /mnt/SeanVolume/CandyDataset/Videos SeanVolume/PublicDataset zfs 1.6T 243k 1.6T 0% /mnt/SeanVolume/PublicDataset SeanVolume/PublicDataset/Applications zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Applications SeanVolume/PublicDataset/Documents zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Documents SeanVolume/PublicDataset/Downloads zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Downloads SeanVolume/PublicDataset/Music zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Music SeanVolume/PublicDataset/Pictures zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Pictures SeanVolume/PublicDataset/Temporary zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Temporary SeanVolume/PublicDataset/Videos zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Videos SeanVolume/SeanDataset zfs 3.9T 2.3T 1.6T 60% /mnt/SeanVolume/SeanDataset SeanVolume/SeanDataset/AppData zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/SeanDataset/AppData SeanVolume/SeanDataset/Applications zfs 1.6T 261k 1.6T 0% /mnt/SeanVolume/SeanDataset/Applications SeanVolume/SeanDataset/Documents zfs 3.0T 1.5T 1.6T 48% /mnt/SeanVolume/SeanDataset/Documents SeanVolume/SeanDataset/Downloads zfs 2.2T 647G 1.6T 29% /mnt/SeanVolume/SeanDataset/Downloads SeanVolume/SeanDataset/Music zfs 1.6T 411k 1.6T 0% /mnt/SeanVolume/SeanDataset/Music SeanVolume/SeanDataset/Pictures zfs 1.6T 2.2M 1.6T 0% /mnt/SeanVolume/SeanDataset/Pictures SeanVolume/SeanDataset/Saved Games zfs 1.6T 243k 1.6T 0% /mnt/SeanVolume/SeanDataset/Saved Games SeanVolume/SeanDataset/Temporary zfs 1.6T 533k 1.6T 0% /mnt/SeanVolume/SeanDataset/Temporary SeanVolume/SeanDataset/Videos zfs 2.4T 850G 1.6T 35% /mnt/SeanVolume/SeanDataset/Videos SeanVolume/jails zfs 1.6T 710M 1.6T 0% /mnt/SeanVolume/jails SeanVolume/jails/.warden-template-pluginjail zfs 1.6T 619M 1.6T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail SeanVolume/jails/.warden-template-pluginjail-9.3 zfs 1.6T 619M 1.6T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3 SeanVolume/jails/pluginjail zfs 1.6T 660M 1.6T 0% /mnt/SeanVolume/jails/pluginjail SeanVolume/ownCloudDataset zfs 1.8T 259G 1.6T 14% /mnt/SeanVolume/ownCloudDataset SeanVolume/.system zfs 1.6T 3.4M 1.6T 0% /var/db/system SeanVolume/.system/cores zfs 1.6T 1.4M 1.6T 0% /var/db/system/cores SeanVolume/.system/samba4 zfs 1.6T 6M 1.6T 0% /var/db/system/samba4 SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261 zfs 1.6T 6M 1.6T 0% /var/db/system/syslog-cd1fc29ce94d4a81a24df77359252261 SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261 zfs 1.6T 224k 1.6T 0% /var/db/system/rrd-cd1fc29ce94d4a81a24df77359252261 devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/pluginjail/dev procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/pluginjail/proc SeanVolume/jails/customplugin_2 zfs 1.6T 1.4G 1.6T 0% /mnt/SeanVolume/jails/customplugin_2 SeanVolume/SeanDataset/ISOImage zfs 1.6T 318k 1.6T 0% /mnt/SeanVolume/SeanDataset/ISOImage devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/customplugin_2/dev procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/customplugin_2/proc /mnt/SeanVolume/ownCloudDataset nullfs 1.8T 259G 1.6T 14% /mnt/SeanVolume/jails/customplugin_2/media
运行清理状态(Web GUI):
Scrub status: Completed Errors: 0 Repaired: 0 Date: Thu Jul 16 07:48:34 2015
我可以做些什么来尝试解决这个问题(除了重启,因为它在线运行ownCloud插件)?
PS:2015/07/17加了:
[root@freenas] ~# zfs list NAME USED AVAIL REFER MOUNTPOINT SeanVolume 10.7T 1.34T 803K /mnt/SeanVolume SeanVolume/.system 304M 1.34T 3.21M legacy SeanVolume/.system/cores 5.53M 1.34T 1.32M legacy SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261 219K 1.34T 219K legacy SeanVolume/.system/samba4 218M 1.34T 5.59M legacy SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261 61.4M 1.34T 5.62M legacy SeanVolume/CandyDataset 1.42T 1.34T 292K /mnt/SeanVolume/CandyDataset SeanVolume/CandyDataset/Applications 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Applications SeanVolume/CandyDataset/Documents 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Documents SeanVolume/CandyDataset/Downloads 1.22T 1.34T 1.22T /mnt/SeanVolume/CandyDataset/Downloads SeanVolume/CandyDataset/Music 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Music SeanVolume/CandyDataset/Pictures 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Pictures SeanVolume/CandyDataset/Temporary 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Temporary SeanVolume/CandyDataset/Videos 206G 1.34T 206G /mnt/SeanVolume/CandyDataset/Videos SeanVolume/PublicDataset 3.03M 1.34T 237K /mnt/SeanVolume/PublicDataset SeanVolume/PublicDataset/Applications 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Applications SeanVolume/PublicDataset/Documents 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Documents SeanVolume/PublicDataset/Downloads 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Downloads SeanVolume/PublicDataset/Music 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Music SeanVolume/PublicDataset/Pictures 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Pictures SeanVolume/PublicDataset/Temporary 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Temporary SeanVolume/PublicDataset/Videos 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Videos SeanVolume/SeanDataset 8.04T 1.34T 2.14T /mnt/SeanVolume/SeanDataset SeanVolume/SeanDataset/AppData 2.64M 1.34T 219K /mnt/SeanVolume/SeanDataset/AppData SeanVolume/SeanDataset/Applications 178G 1.34T 256K /mnt/SeanVolume/SeanDataset/Applications SeanVolume/SeanDataset/Documents 2.58T 1.34T 1.33T /mnt/SeanVolume/SeanDataset/Documents SeanVolume/SeanDataset/Downloads 1.07T 1.34T 661G /mnt/SeanVolume/SeanDataset/Downloads SeanVolume/SeanDataset/ISOImage 103G 1.34T 310K /mnt/SeanVolume/SeanDataset/ISOImage SeanVolume/SeanDataset/Music 51.7G 1.34T 402K /mnt/SeanVolume/SeanDataset/Music SeanVolume/SeanDataset/Pictures 131G 1.34T 2.08M /mnt/SeanVolume/SeanDataset/Pictures SeanVolume/SeanDataset/Saved Games 75.7G 1.34T 237K /mnt/SeanVolume/SeanDataset/Saved Games SeanVolume/SeanDataset/Temporary 9.63G 1.34T 520K /mnt/SeanVolume/SeanDataset/Temporary SeanVolume/SeanDataset/Videos 846G 1.34T 792G /mnt/SeanVolume/SeanDataset/Videos SeanVolume/jails 3.80G 1.34T 678M /mnt/SeanVolume/jails SeanVolume/jails/.warden-template-pluginjail 6.33M 1.34T 591M /mnt/SeanVolume/jails/.warden-template-pluginjail SeanVolume/jails/.warden-template-pluginjail-9.3 597M 1.34T 591M /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3 SeanVolume/jails/customplugin_2 1.17G 1.34T 1.27G /mnt/SeanVolume/jails/customplugin_2 SeanVolume/jails/pluginjail 1.28G 1.34T 630M /mnt/SeanVolume/jails/pluginjail SeanVolume/ownCloudDataset 1.21T 1.34T 255G /mnt/SeanVolume/ownCloudDataset freenas-boot 680M 107G 144K none freenas-boot/ROOT 670M 107G 144K none freenas-boot/ROOT/Initial-Install 8K 107G 659M legacy freenas-boot/ROOT/default 670M 107G 665M legacy freenas-boot/grub 7.76M 107G 7.76M legacy
2015/07/17已添加:
在使用zfs list -t snapshot命令之后,它显示了许多微小的快照(总共2,534个文件…)。
我正在检查,并删除这些快照文件。
删除后,再次运行df -TH !
2015/07/20地址:
我删除了许多快照文件后,我得到了一些空间!
但…
FreeNAS 9.3 Web GUI崩溃…
在清除大量快照(大约1500)后,FreeNAS 9.3 Web GUI将显示“发生错误”。 两天后
[root@freenas] /etc/defaults# df -TH Filesystem Type Size Used Avail Capacity Mounted on freenas-boot/ROOT/default zfs 115G 698M 114G 1% / devfs devfs 1.0k 1.0k 0B 100% /dev tmpfs tmpfs 33M 5.5M 28M 16% /etc tmpfs tmpfs 4.2M 8.2k 4.2M 0% /mnt tmpfs tmpfs 11G 67M 11G 1% /var freenas-boot/grub zfs 114G 8.1M 114G 0% /boot/grub SeanVolume zfs 4T 822k 4T 0% /mnt/SeanVolume SeanVolume/CandyDataset zfs 4T 299k 4T 0% /mnt/SeanVolume/CandyDataset SeanVolume/CandyDataset/Downloads zfs 5.3T 1.3T 4T 25% /mnt/SeanVolume/CandyDataset/Downloads SeanVolume/CandyDataset/Videos zfs 4.2T 221G 4T 5% /mnt/SeanVolume/CandyDataset/Videos SeanVolume/PublicDataset zfs 4T 224k 4T 0% /mnt/SeanVolume/PublicDataset SeanVolume/SeanDataset zfs 9.6T 5.6T 4T 59% /mnt/SeanVolume/SeanDataset SeanVolume/jails zfs 4T 710M 4T 0% /mnt/SeanVolume/jails SeanVolume/jails/.warden-template-pluginjail zfs 4T 619M 4T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail SeanVolume/jails/.warden-template-pluginjail-9.3 zfs 4T 619M 4T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3 SeanVolume/jails/pluginjail zfs 4T 661M 4T 0% /mnt/SeanVolume/jails/pluginjail SeanVolume/ownCloudDataset zfs 4.3T 336G 4T 8% /mnt/SeanVolume/ownCloudDataset SeanVolume/.system zfs 4T 3.4M 4T 0% /var/db/system SeanVolume/.system/cores zfs 4T 1.4M 4T 0% /var/db/system/cores SeanVolume/.system/samba4 zfs 4T 6.1M 4T 0% /var/db/system/samba4 SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261 zfs 4T 11M 4T 0% /var/db/system/syslog-cd1fc29ce94d4a81a24df77359252261 SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261 zfs 4T 224k 4T 0% /var/db/system/rrd-cd1fc29ce94d4a81a24df77359252261 devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/pluginjail/dev procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/pluginjail/proc SeanVolume/jails/customplugin_2 zfs 4T 1.4G 4T 0% /mnt/SeanVolume/jails/customplugin_2 devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/customplugin_2/dev procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/customplugin_2/proc /mnt/SeanVolume/ownCloudDataset nullfs 4.3T 336G 4T 8% /mnt/SeanVolume/jails/customplugin_2/media
我试着总结给出的信息,并给出一些解释(我知道,它是什么意思):
关于zpool列表输出:
SIZE是所有光盘的大小,包括用于冗余的光盘(所以加上zpool状态的输出,也就是说,您已经将8个3TB驱动器放在一个RAID-Z3configuration中)
ALLOC是已使用的空间,也包括用于冗余的光盘(因为8个光盘中的3个用于冗余,所以至less比实际数据大小高(5 + 3)/ 5 = 1.6倍)
Free是可用空间,也包括用于冗余的光盘(所以可用空间最多为5 /(5 + 3)= 0,625所示的自由空间)
zfs列表命令说,有1,34T可用, zpool列表说,3,16T空闲,减去冗余空间,我期望约1,975T。
我知道,zfs需要一些元数据空间,但是在我的linux系统中,RAID-Z1中有4个2TB的驱动器,差异大约是6%,这里的差别大概是47%,我不能说差异来自哪里。
关于df -Th的输出
df没有findzfs快照,所以不能计算它们,所以使用的空间不是zfs文件系统中使用的空间。 对于zfs文件系统中的分配空间,使用zfs列表
删除快照可能需要一段时间,因为您发现了,似乎是由devise,你可以看看http://nex7.blogspot.de/2013/03/readme1st.html
删除快照后,如果我计算你的df输出,你的池中大约有7,2T的数据,大概有4T,所有的都是11,2T。
如果我把你的完整尺寸(21,8T)减去冗余盘,我预计大约5 /(5 + 3)* 21,8T = 13,6T(相差大约20%)
我认为20%对于zfs所需要的元数据有一点点的影响,但我不能告诉你,它在哪里丢失。 也许你有很多非常小的文件,它们使用的元数据空间比没有那么多的大文件要多。
我很抱歉,但是我并没有深入zfs的内部,我可以告诉你,在哪里获得元数据空间的使用。
您可以再次尝试一个zfs列表 ,并将已使用的空间与您从df获得的已用空间进行比较。 也许它提供了一个提示。
编辑21.07.2015
关于大小:3T驱动器实际上有2.7TB(因为驱动器制造商使用1TB = 1.000.000.000.000字节,其中计算机使用1TiB = 1 * 1024 * 1024 * 1024 * 1024 = 1.099.511.627.776字节。
2,7 * 8驱动器= 21,8T
在正常的RAID系统中,最小的驱动器决定了较大驱动器的可用大小。