我最近安装了一个新的磁盘并在其上创build了一个zpool:
/# zpool create morez /dev/sdb
使用一段时间后,我注意到它很慢:
/morez# fio --name rw --rw rw --size 10G read: IOPS=19.6k, BW=76.6MiB/s (80.3MB/s)(5120MiB/66834msec) write: IOPS=19.6k, BW=76.6MiB/s (80.3MB/s)(5120MiB/66834msec)
这个testing与我的实际使用情况非常相似。 我正在从磁盘读取中等数量(〜10k)的图像(每个〜2个MiB)。 当磁盘大部分是空的时候,它们都是一次写入的,所以我不认为它们会被分割。
为了比较,我testing了ext4:
/# gdisk /dev/sdb ... /# mkfs.ext4 -f /dev/sdb1 && mount /dev/sdb1 /mnt && cd /mnt /mnt# fio --name rw --rw rw --size 10G read: IOPS=48.3k, BW=189MiB/s (198MB/s)(5120MiB/27135msec) write: IOPS=48.3k, BW=189MiB/s (198MB/s)(5120MiB/27135msec)
和btrfs:
/# mkfs.btrfs -f /dev/sdb1 && mount /dev/sdb1 /mnt && cd /mnt /mnt# fio --name rw --rw rw --size 10G read: IOPS=51.3k, BW=201MiB/s (210MB/s)(5120MiB/25528msec) write: IOPS=51.3k, BW=201MiB/s (210MB/s)(5120MiB/25528msec)
什么可能导致ZFS的性能问题,我怎样才能让它更快?
我也尝试明确设置zpool的扇区大小,因为我的磁盘( 希捷ST1000DM003 )使用4096字节的物理扇区:
/# zpool create -o ashift=12 morez /dev/sdb
这并没有改善性能:
/morez# fio --name rw --rw rw --size 10G read: IOPS=21.3k, BW=83.2MiB/s (87.2MB/s)(5120MiB/61573msec) write: IOPS=21.3k, BW=83.2MiB/s (87.2MB/s)(5120MiB/61573msec)
奇怪的是,使用zvol有很好的性能:
/# zfs create -V 20G morez/vol /# fio --name rw --filename /dev/zvol/morez/vol --rw rw --size 10G read: IOPS=52.7k, BW=206MiB/s (216MB/s)(5120MiB/24852msec) write: IOPS=52.7k, BW=206MiB/s (216MB/s)(5120MiB/24852msec)
为什么这只影响ZFS文件系统而不影响zvols?
在评论中,有人build议这种差异可能是由于caching。 经过进一步的testing,我不相信是这样。 我将btrfstesting的大小增加到远远高于我的计算机的内存量,其性能仍然明显高于ZFS:
/# mkfs.btrfs -f /dev/sdb1 && mount /dev/sdb1 /mnt && cd /mnt /mnt# $ fio --name rw --rw rw --size 500G --runtime 3600 --time_based --ramp_time 900 read: IOPS=41.9k, BW=164MiB/s (172MB/s)(576GiB/3600003msec) write: IOPS=41.9k, BW=164MiB/s (172MB/s)(576GiB/3600003msec)
在运行fio之前,以下是ZFS属性的外观。 这些只是使用默认设置创buildzpool的结果。
# zpool get all morez NAME PROPERTY VALUE SOURCE morez size 928G - morez capacity 0% - morez altroot - default morez health ONLINE - morez guid [removed] default morez version - default morez bootfs - default morez delegation on default morez autoreplace off default morez cachefile - default morez failmode wait default morez listsnapshots off default morez autoexpand off default morez dedupditto 0 default morez dedupratio 1.00x - morez free 928G - morez allocated 276K - morez readonly off - morez ashift 0 default morez comment - default morez expandsize - - morez freeing 0 default morez fragmentation 0% - morez leaked 0 default morez feature@async_destroy enabled local morez feature@empty_bpobj enabled local morez feature@lz4_compress active local morez feature@spacemap_histogram active local morez feature@enabled_txg active local morez feature@hole_birth active local morez feature@extensible_dataset enabled local morez feature@embedded_data active local morez feature@bookmarks enabled local morez feature@filesystem_limits enabled local morez feature@large_blocks enabled local # zfs get all morez NAME PROPERTY VALUE SOURCE morez type filesystem - morez creation Thu Jun 29 19:34 2017 - morez used 240K - morez available 899G - morez referenced 96K - morez compressratio 1.00x - morez mounted yes - morez quota none default morez reservation none default morez recordsize 128K default morez mountpoint /morez default morez sharenfs off default morez checksum on default morez compression off default morez atime on default morez devices on default morez exec on default morez setuid on default morez readonly off default morez zoned off default morez snapdir hidden default morez aclinherit restricted default morez canmount on default morez xattr on default morez copies 1 default morez version 5 - morez utf8only off - morez normalization none - morez casesensitivity sensitive - morez vscan off default morez nbmand off default morez sharesmb off default morez refquota none default morez refreservation none default morez primarycache all default morez secondarycache all default morez usedbysnapshots 0 - morez usedbydataset 96K - morez usedbychildren 144K - morez usedbyrefreservation 0 - morez logbias latency default morez dedup off default morez mlslabel none default morez sync standard default morez refcompressratio 1.00x - morez written 96K - morez logicalused 72.5K - morez logicalreferenced 40K - morez filesystem_limit none default morez snapshot_limit none default morez filesystem_count none default morez snapshot_count none default morez snapdev hidden default morez acltype off default morez context none default morez fscontext none default morez defcontext none default morez rootcontext none default morez relatime off default morez redundant_metadata all default morez overlay off default
被警告 – 你的工作缺乏direct=1
( http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-direct )所以一些数量的I / O正在执行(读取和写入)可能会被操作系统本身caching,从而通过虚高的方式来扭曲你的结果。