速度更快的服务器上创build速度更慢 为什么？

我有两台服务器。 旧的是24GB内存，i7-950 @ 3,0Ghz，2x 1,5TB SATA硬盘（ST31500341AS）。 新的32GB RAM，i7-3770 @ 3,4GHz，2x 3,0 TB SATA硬盘（ST3000DM001-9YN166）。

现在我没有想到，在新主机上创build一个新的虚拟机需要的时间是旧主机的两倍 ，尽pipe新 CPU比旧的速度快66％，而且内存也更多。 而且在最初的raid同步中harddrives也更快，但是我没有数字。

所以我怀疑硬盘是限制因素，我在两个系统上运行Bonnie ++。 这两个系统运行Ubuntu 12.04和testing发生在一个lvm卷上，使用xfs启用延迟计数器。

旧服务器的结果:(同时正在pipe理10个生产Xen虚拟机）。

• EBS快照是否会大大减慢EBS的性能？
• Apache日志logging性能
• Windows DNSdebugging日志logging性能影响
• 如何使用Nagios在无法访问/停机期间收集服务器的性能数据？
• 如何过滤命令列的顶部输出？

Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 8 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP 1432M 499 97 57940 11 36990 10 657 98 64289 9 195.1 6 Latency 31917us 35567us 340ms 40027us 162ms 445ms Version 1.96 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 256 6232 26 12543 19 5888 23 4247 17 317 0 368 1 Latency 1199ms 95254us 1804ms 2313ms 1183ms 12470ms 

新服务器的结果：（同时不忙pipe理1个空闲的备份虚拟机）

 Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 8 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP 11G 447 96 93517 14 59222 18 644 97 144128 21 310.6 11 Latency 58753us 299ms 575ms 45753us 292ms 238ms Version 1.96 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 256 4838 16 321243 100 4927 12 4601 16 309249 99 405 1 Latency 3598ms 466us 8146ms 2342ms 58us 13765ms 

我认为这清楚地表明， 新服务器上的读/写性能确实比旧服务器上的好很多。 （可能是因为更快的硬盘驱动器或旧服务器上的负载）。 然而，文件创build（和删除）要慢得多。

我能做些什么来改善或分析这个？

这是旧服务器的lspci输出：

 00:00.0 Host bridge: Intel Corporation 5520/5500/X58 I/O Hub to ESI Port (rev 13) 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13) 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13) 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13) 00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management Registers (rev 13) 00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13) 00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13) 00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 13) 00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1 00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5 00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) 00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller 02:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI RV620 LE [Radeon HD 3450] 02:00.1 Audio device: Advanced Micro Devices [AMD] nee ATI RV620 HDMI Audio [Radeon HD 3400 Series] 07:00.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05) ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05) ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05) ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05) ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05) ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05) ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05) ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05) ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05) ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05) ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05 ) ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05) ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05) ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05) ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05 ) ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05) ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05) ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05) ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05 ) 

旧服务器上的lspci -vvv sata控制器部分：

 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller (prog-if 01 [AHCI 1.0]) Subsystem: Micro-Star International Co., Ltd. Device 7522 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 324 Region 0: I/O ports at a000 [size=8] Region 1: I/O ports at 9c00 [size=4] Region 2: I/O ports at 9880 [size=8] Region 3: I/O ports at 9800 [size=4] Region 4: I/O ports at 9480 [size=32] Region 5: Memory at fbafa000 (32-bit, non-prefetchable) [size=2K] Capabilities: [80] MSI: Enable+ Count=1/16 Maskable- 64bit- Address: fee4000c Data: 4121 Capabilities: [70] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004 Capabilities: [b0] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: ahci 

这是新服务器的lspci输出：

 00:00.0 Host bridge: Intel Corporation Ivy Bridge DRAM Controller (rev 09) 00:01.0 PCI bridge: Intel Corporation Ivy Bridge PCI Express Root Port (rev 09) 00:02.0 VGA compatible controller: Intel Corporation Ivy Bridge Graphics Controller (rev 09) 00:14.0 USB controller: Intel Corporation Panther Point USB xHCI Host Controller (rev 04) 00:16.0 Communication controller: Intel Corporation Panther Point MEI Controller #1 (rev 04) 00:1a.0 USB controller: Intel Corporation Panther Point USB Enhanced Host Controller #2 (rev 04) 00:1c.0 PCI bridge: Intel Corporation Panther Point PCI Express Root Port 1 (rev c4) 00:1c.4 PCI bridge: Intel Corporation Panther Point PCI Express Root Port 5 (rev c4) 00:1d.0 USB controller: Intel Corporation Panther Point USB Enhanced Host Controller #1 (rev 04) 00:1f.0 ISA bridge: Intel Corporation Panther Point LPC Controller (rev 04) 00:1f.2 SATA controller: Intel Corporation Panther Point 6 port SATA Controller [AHCI mode] (rev 04) 00:1f.3 SMBus: Intel Corporation Panther Point SMBus Controller (rev 04) 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 09) 

新服务器上的lspci -vvv sata控制器部分：

 00:1f.2 SATA controller: Intel Corporation Panther Point 6 port SATA Controller [AHCI mode] (rev 04) (prog-if 01 [AHCI 1.0]) Subsystem: ASUSTeK Computer Inc. Device 84ca Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 323 Region 0: I/O ports at f0b0 [size=8] Region 1: I/O ports at f0a0 [size=4] Region 2: I/O ports at f090 [size=8] Region 3: I/O ports at f080 [size=4] Region 4: I/O ports at f060 [size=32] Region 5: Memory at f7c12000 (32-bit, non-prefetchable) [size=2K] Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee00338 Data: 0000 Capabilities: [70] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004 Capabilities: [b0] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: ahci 

两个testing分区的xfs_info结果是二进制相同的：

 # xfs_info /mnt meta-data=/dev/xvda3 isize=256 agcount=4, agsize=3276800 blks = sectsz=512 attr=2 data = bsize=4096 blocks=13107200, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=6400, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 

现在我在两台机器上的相同虚拟机上重复testing。 旧服务器上虚拟机的结果：

 Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP vm.sh-solutions. 2G 454 97 63287 9 17729 4 644 98 64923 8 120.5 2 Latency 99698us 55973us 602ms 50737us 242ms 1851ms Version 1.96 ------Sequential Create------ --------Random Create-------- vm.sh-solutions.de -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 256 7996 36 248222 98 16188 51 9244 37 263652 99 588 2 Latency 666ms 200us 491ms 636ms 77us 9010ms 

新服务器上虚拟机的结果：

 Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP vm.sh-solutions. 2G 760 96 152604 19 71291 16 1084 97 214864 23 453.0 7 Latency 23365us 42943us 442ms 55716us 42838us 124ms Version 1.96 ------Sequential Create------ --------Random Create-------- vm.sh-solutions.de -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 256 4215 11 403505 97 7103 13 3637 9 405289 99 481 1 Latency 2746ms 115us 1819ms 4146ms 22us 17072ms 

新服务器再一次具有更高的吞吐量，但创build（和删除）的速度要慢得多。

旧服务器parted /dev/sda print ：

 Model: ATA ST31500341AS (scsi) Disk /dev/sda: 1500GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 4 16.5MB 38.9GB 38.9GB primary lvm 1 38.9GB 49.2GB 10.2GB primary xfs raid 2 49.2GB 59.4GB 10.2GB primary xfs boot, raid 3 59.4GB 1500GB 1441GB primary raid 

新服务器parted /dev/sda print ：

 Model: ATA ST3000DM001-9YN1 (scsi) Disk /dev/sda: 3001GB Sector size (logical/physical): 512B/4096B Partition Table: gpt Number Start End Size File system Name Flags 1 1049kB 3146kB 2097kB grub bios_grub 2 3146kB 68.7GB 68.7GB swap raid 3 68.7GB 79.5GB 10.7GB xfs root1 raid 4 79.5GB 90.2GB 10.7GB xfs root2 raid 5 90.2GB 3001GB 2910GB raid raid 

在创build了带有mkfs.xfs -d su = 4k，sw = 1 -l su = 4k`的fs之后，在同一个VM中的新服务器上得到结果。 （ 没有改进 ）

 Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP vm.sh-solutions. 2G 764 95 160175 21 71847 17 669 97 215201 23 444.1 8 Latency 20880us 46904us 327ms 31263us 50802us 137ms Version 1.96 ------Sequential Create------ --------Random Create-------- vm.sh-solutions.de -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 256 4180 13 257543 100 7547 17 4372 12 377933 99 410 1 Latency 1739ms 160us 1774ms 2073ms 79us 9058ms 

更新 ：到目前为止，我把所有的虚拟机移动到新的服务器，让我重新安装旧的服务器与新服务器的一切版本。 我在那台机器上重复了vmtesting：

 Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP vm.sh-solutions. 2G 701 97 110157 14 27335 6 1034 98 163694 17 341.1 5 Latency 22008us 118ms 1481ms 30413us 26635us 742ms Version 1.96 ------Sequential Create------ --------Random Create-------- vm.sh-solutions.de -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 256 9775 34 389576 100 19943 50 10919 37 374190 99 599 2 Latency 900ms 164us 356ms 533ms 26us 8015ms 

注意 ：在这台旧主机上，没有其他虚拟机在该主机上处于活动状态。 它每秒重新创build10000次操作。 新主机（也没有其他活动虚拟机）的相同（较早）testing导致在新服务器上仅创build4215个 。 即使当旧的服务器方式“忙”达到了6200创build…这使得这比预期的更糟糕…

更新 ：从现在起，我再也无法访问旧的服务器了。 我试图让我的手在类似的机器，但。

在新服务器上的cat /proc/mounts有（旁边）：

 /dev/xvda3 /mnt2 xfs rw,relatime,attr2,delaylog,sunit=8,swidth=8,noquota 0 0 

 Sector size (logical/physical): 512B/512B Sector size (logical/physical): 512B/4096B